summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2017-11-20EC2: Fix bug using fallback_nic and metadata when restoring from cache.Scott Moser
If user upgraded to new cloud-init and attempted to run 'cloud-init init' without rebooting, cloud-init restores the datasource object from pickle. The older version pickled datasource object had no value for _network_config or fallback_nic. This caused the Ec2 datasource to attempt to reconfigure networking with a None fallback_nic. The pickled object also cached an older version of ec2 metadata which didn't contain network information. This branch does two things: - Add a fallback_interface property to DatasourceEC2 to support reading the old .fallback_nic attribute if it was set. New versions will call net.find_fallback_nic() if there has not been one found. - Re-crawl metadata if we are on Ec2 and don't have a 'network' key in metadata LP: #1732917
2017-11-19EC2: Kill dhclient process used in sandbox dhclient.Scott Moser
dhclient runs, obtains a address and then backgrounds itself. cloud-init did not take care to kill it after it was done with it. After it has run and created the leases, we can kill it. LP: #1732964
2017-11-16ntp: fix configuration template rendering for openSUSE and SLESChad Smith
Add opensuse distro support to cc_ntp module. LP: #1726572
2017-11-16centos: Provide the failed #include url in error messagesChad Smith
On python 2.7 and earlier (CentOS 6 & 7), UrlErrors raised by requests do not report the url which failed. In such cases, append the url if not present in the error message. This fixes nightly CI failures at https://jenkins.ubuntu.com/server/view/cloud-init/.
2017-11-13Catch UrlError when #include'ing URLsAndrew Jorgensen
Without this the entire stage can fail, which will leave an instance unaccessible. Reviewed-by: Tom Kirchner <tjk@amazon.com> Reviewed-by: Matt Nierzwicki <nierzwic@amazon.com> Reviewed-by: Ben Cressey <bcressey@amazon.com>
2017-11-10hosts: Fix openSUSE and SLES setup for /etc/hosts and clarify docs.Robert Schweikert
The etc/hosts file is was not properly setup for openSUSE or SLES when manage_etc_hosts is set in the config file. Improve the doc to address the fact that the 'localhost' ip is distribution dependent (not always 127.0.0.1). LP: #1731022
2017-11-10rh_subscription: Perform null checks for enabled and disabled repos.Dave Mulford
The rh_subscription module doesn't perform null checks when attempting to iterate on the enabled and disable repos arrays. When only one is specified, cloud-init fails to run.
2017-11-10Improve warning message when a template is not found.Robert Schweikert
At present the location for the template file look up upon failure includes the template file itself. However based on the wording of the message it should only contain the template directory issue LP: #1731035
2017-11-09Replace the temporary i9n.brickies.net with i9n.cloud-init.io.Scott Moser
We had used some dns records in i9n.brickies.net (my personal domain) as a temporary solution until we got names registered in the cloud-init.io namespace. We now have CNAME records for: ubuntu.i9n.cloud-init.io cloudinit1.cloud-init.io cloudinit2.cloud-init.io
2017-11-09Azure: don't generate network configuration for SRIOV devicesScott Moser
Azure kernel now configures the SRIOV devices itself so cloud-init does not need to provide any SRIOV device configuration or udev naming rules. LP: #1721579
2017-11-07tests: address some minor feedback missed in last merge.Scott Moser
3 things here: a.) link to a bug that we opened to track what made us add dns entries for hostname of our guests. b.) spelling fix. c.) raise an instance of a NotImplementedError not the class.
2017-11-06tests: integration test cleanup and full pass of nocloud-kvm.Scott Moser
Integration test harness changes: * Enable collection of console log in nocloud-kvm and lxd. * Collect the console log to results for all test runs. * change 'tmpfile' to pick name locally instead of using 'mktemp'. * drop the 'instance' attribute from nocloud-kvm Image and demote LXDImage.instance to a private attribute. This is because Images do not actually have instances. (LXDImage internally uses a booted system to modify the image). * Add 'TargetBase' as a superclass of Image and Instance providing implementations of execute, read_data, write_data, pull_file, and push_file. These all depend on an implementation of _execute. * Improve '_execute' implementations to support accepting stdin. * execute supports 'rcs=False' meaning 'do not raise exception'. * Drop support for pylxd < 2.2. older versions cannot determine exit code of 'execute', which makes them unusable. * make NoCloudKVMInstance._execute run as root via sudo. This required some changes so that 'hostname' could be reverse-looked up in order to avoid sudo taking a long time (~20 seconds). * re-use existing ssh connection in nocloud-kvm. Test changes here: * do not use /tmp, but rather /var/tmp (LP: #1707222) * make keys_to_console assertions more strict. * change user test cases to always add default (ubuntu) user so that nocloud-kvm's execute which operates over ssh can work.
2017-10-31Gentoo: chmod +x on all files in sysvinit/gentoo/ckonstanski
Add execute bit for gentoo sysvinit scripts. LP: #1727126
2017-10-31EC2: Limit network config to fallback nic, fix local-ipv4 only instances.Chad Smith
VPC instances have the option to specific local only IPv4 addresses. Allow Ec2Datasource to enable dhcp4 on instances even if local-ipv4s is configured on an instance. Also limit network_configuration to only the primary (fallback) nic. LP: #1728152
2017-10-26Gentoo: Use "rc-service" rather than "service".ckonstanski
The "service" command will be disappearing soon from Gentoo"s openrc package. The "rc-service" command is preferred. LP: #1727121
2017-10-23resizefs: Fix regression when system booted with root=PARTUUID=Chad Smith
A recent cleanup of the resizefs module broke resizing when a system was booted with root=PARTUUID=<uuid> and the device /dev/root does not exist. This path is exposed with the Ubuntu 16.04 but not with Ubuntu 17.10. A recreate exists under bug 1684869. LP: #1725067
2017-10-23tools: make yum package installation more reliableScott Moser
During continuous integration tests, we're seeing quite a lot of unreliablity when running 'yum install'. The change here is to move to re-trying a run of 'yum install --downloadonly' for 10 times or until it succeeds. Then afterwards, running yum install from the cache. This seems safer in general than just re-trying an install operation, since we are specifically affected by the download phase failing. Also present are some flake8 fixes to tools/read-dependencies.
2017-10-20citest: fix remaining warnings raised by integration tests.Scott Moser
There was fallout in a full integration test run from my adding of test_no_warnings_in_log which asserted that there could not be a WARNING found in the /var/log/cloud-init.log This fixes 2 of the cases: * TestCommandOutputSimple had a valid WARNING written, so adjust its test case to allow for that. * TestLxdDir had a valid config in the test but the module would log a WARNING, so fix the module. Also updates lxd unit tests to look for WARN themselves.
2017-10-20citest: show the class actual class name in results.Scott Moser
Tests are currently run by creating a temporary subclass of each class and then executing it (in get_suites). When running the tests suite the output would contain the temporary class name. That was less than useful, and made batch runs almost impossible to identify which test case had an error. This change goes from output of: FAIL: test_no_warnings_in_log \ (tests.cloud_tests.testcases.get_suite.<locals>.tmp) To FAIL: test_no_warnings_in_log \ (tests.cloud_tests.testcases.modules.ntp.TestNtp)
2017-10-20ntp: fix config module schema to allow empty ntp configChad Smith
Fix three things related to the ntp module: 1. Fix invalid cloud-config schema in the integration test which provided empty dicts instead of emptylists for pools and servers 2. Correct logic in the ntp module to allow support for the minimal cloud-config 'ntp:' without raising a RuntimeError. Docs and schema definitions already describe that cloud-config's ntp can be empty. An ntp configuration with neither pools nor servers will be configured with a default set of ntp pools. As such, the ntp module now officially allows the following ntp cloud-configs: - ntp: - ntp: {} - ntp: servers: [] pools: [] 3. Add a simple unit test which validates all cloud-config provided to our integration tests to ensure it adheres to any defined module schema so as more jsonschema definitions are added, we validate our integration test configs. LP: #1724951
2017-10-19tools: disable fastestmirror if using proxyJoshua Powers
Per centos documentation using the fastestmirror plugin is effective at finding the fastest mirror, unless you are behind a proxy. In that case you should disable it. Therefore, in our tests if we are setting the proxy we should also disable the fastestmirror plugin.
2017-10-18schema: Log debug instead of warning when jsonschema is not available.Scott Moser
When operating in expected path, cloud-init should avoid logging with warning. That causes 'WARNING' messages in /var/log/cloud-init.log. By default, warnings also go to the console. Since jsonschema is a optional dependency, and not present on xenial and zesty, cloud-init should not warn there. Also here: * Add a test to integration tests to assert that there are no warnings in /var/log/cloud-init.log. * Update one integration test that did show warning and the related documentation and examples. LP: #1724354
2017-10-10simpletable: Fix get_string method to return table-formatted stringChad Smith
Output in cloud-init-output.log contained only the string representation of a SimpleTable object instead of the table formatted content. This bug also affected ssh_authkey_fingerprints. LP: #1722566
2017-10-05net: Handle bridge stp values of 0 and convert to boolean typeChad Smith
Update unit tests to pass a 0 instead of 'off' to validate that network state is properly written.
2017-10-05tools: Give specific --abbrev=8 to "git describe"Scott Moser
The tools that use "git describe" were just assuming a consisent number of characters in the hash. It seems ubuntu 16.04 would use 7 and later versions use 8. To avoid that discrepency in developer environments, set it to 8.
2017-10-05network: bridge_stp value not always correctRyan Harper
Update network_state to store the bridge_stp value as a boolean. The various renderers then can map the boolean value to the correct output as needed; eni uses 'on/off', sysconfig uses 'yes/no' and netplan will use the boolean directly. Update unittest values for sysconfig and netplan. Both contained the network_state string value which resulted in not correctly enable/disable STP in the target system. Update network_state comment (fd -> forward-delay, add stp as boolean) on bridge commands to match the expected format of a netplan bridge command. LP: #1721157
2017-10-03tests: re-enable tox with nocloud-kvm supportJoshua Powers
With the addition of the nocloud-kvm support a few other python modules were pulled in as required and as a result this broke the tox run. The fix was to add paramiko and simplestreams to re-enable testing.
2017-10-03systemd: remove limit on tasks created by cloud-init-final.service.Robert Schweikert
Depending on distribution the default number of tasks (threads) maybe unexpectedly low or it may be the default systemd setting (512). Setting TasksMax to "infinity" in cloud-init-final.service removes the restriction on tasks created. LP: #1717969
2017-10-03suse: Support addition of zypper repos via cloud-config.Robert Schweikert
This adds a config module so support for adding zypper repositories via cloud-config. LP: #1718675
2017-10-03tests: Combine integration configs and testcasesJoshua Powers
Combine the configs and testcases directories, so all files are together in one place. Update the test config location as well.
2017-10-03Azure, CloudStack: Support reading dhcp options from systemd-networkd.Dimitri John Ledkov
Systems that used systemd-networkd's dhcp client would not be able to get information on the Azure endpoint (placed in Option 245) or the CloudStack server (in 'server_address'). The change here supports reading these files in /run/systemd/netif/leases. The files declare that "This is private data. Do not parse.", but at this point we do not have another option. LP: #1718029
2017-10-02packages/debian/copyright: remove mention of boto and MIT licenseScott Moser
boto_utils.py had been removed some time ago, and the current cloudinit/ec2_utils.py is not based on what was in boto_utils. We just failed to remove the mention of it from the upstream debian/copyright. And then put it back in everywhere in recent changes to get upstream and ubuntu in sync.
2017-10-02systemd: only mention Before=apt-daily.service on debian based distros.Robert Schweikert
Ordering on apt service should only be set up on Debian based distributions. This changes is really a net-zero in runtime result. But, mentioning apt on a rpm based distro could be confusing.
2017-10-02Add missing simpletable and simpletable tests for failed mergeChad Smith
2017-10-02Remove prettytable dependency, introduce simpletableAndrew Jorgensen
The first revision of this rendered tables with less decoration but there was a desire upstream to avoid possibly breaking some parsing someone might be doing, so it has been revised to render the same as prettytable for the cases cloud-init actually uses.
2017-09-29debian/copyright: dep5 updates, reorganize, add Apache 2.0 license.Joshua Powers
The copyright was updated to be lintian clean and reorganized to list the licenses at the bottom after declaring the metadata and file information. Add the MIT license to the file. LP: #1718681
2017-09-25tests: remove dependency on shlexJoshua Powers
This removes shlex and converts the subprocess commands to use a list over a string.
2017-09-25AltCloud: Trust PATH for udevadm and modprobe.Scott Moser
Previously we had hard coded paths in /sbin for the udevadm and modprobe programs invoked by AltCloud. Its more flexible to expect the PATH to be set correctly. Debian: #852564
2017-09-22DataSourceOVF: use util.find_devs_with(TYPE=iso9660)Ryan Harper
DataSourceOVF attempts to find iso files via walking os.listdir('/dev/') which is far too wide. This approach is too invasive and can sometimes race with systemd attempting to fsck and mount devices. Instead, utilize cloudinit.util.find_devs_with to filter devices by criteria (which uses blkid under the covers). This results in fewer attempts to mount block devices which do not contain iso filesystems. Unittest changes include: - cloudinit.tests.helpers; introduce add_patch() helper - Add unittest coverage for DataSourceOVF use of transport_iso9660 LP: #1718287
2017-09-21tests: remove a temp file used in bootcmd tests.Scott Moser
The bootcmd test was leaving files in the tmpdir named ci-FakeExtendedTempFile.XXXXXX. This cleans those up.
2017-09-21release 17.1Scott Moser
Bump the version in cloudinit/version.py to be 17.1 and update ChangeLog.
2017-09-21doc: document GCE datasource.Arnd Hannemann
Add some minimal documentation for GCE datasource.
2017-09-21suse: updates to templates to support openSUSE and SLES.Robert Schweikert
Things done here: - identify 'suse' as a variant in util.system_info and also tools/render-cloudcfg. - update systemd and cloud.cfg templates for suse specific changes. LP: #1718640
2017-09-21suse: Copy sysvinit files from redhat with slight changes.Robert Schweikert
Here we commit the SuSE provided sysvinit scripts. They are very similar to those in redhat/ directory. They differ in small but important ways. Rather than build a template system here we will just accept the copy and paste. sysvinit in both RedHat and SuSE is EOL, so we do not expect any real maintenance cost here. LP: #1718649
2017-09-20docs: fix sphinx module schema documentationChad Smith
Create a copy of each modules schema attribute when generating sphinx docs to avoid altering the actual module dict in memory. This avoids illegible rendering of module examples and distros where each character of a list was represented on a separate line by itself. Fixes ntp, resizefs, runcmd and bootcmd docs.
2017-09-20tests: Add cloudinit package to all test targetsChad Smith
The package cloudinit was sparsely added to only the makefile's unittest target and tox's py3 target. This branch adds cloudinit package to 'make unittest3' and all tox environments. It tweaks one cloudinit unit test to use mocked_object.call_count instead of mocked_object.assert_called_once which is not defined in some python unittest versions.
2017-09-20Makefile: No longer look for yaml files in obsolete ./bin/.Scott Moser
The bin/ dir was deleted some time ago, but the Makefile was still searching for files down it. This didn't cause any problems other Than a wierd looking error message in a build log.
2017-09-19tests: fix ds-identify unit tests to set EC2_STRICT_ID_DEFAULT.Scott Moser
The variable DI_EC2_STRICT_ID_DEFAULT was not being set in unit tests so when 16.04 built, which changed that setting in patches the tests would unexpectedly fail.
2017-09-18ec2: Fix maybe_perform_dhcp_discovery to use /var/tmp as a tmpdirChad Smith
/run/cloud-init/tmp is on a filesystem mounted noexec, so running dchlient in Ec2Local during discovery breaks with 'Permission denied'. This branch allows us to run from a different tmp dir so we have exec rights. LP: #1717627
2017-09-18Azure: wait longer for SSH pub keys to arrive.Paul Meyer
Currently the Azure data source waits up to 60 seconds. This has proven not to be sufficient to provide resiliency to unrelated transient failures in other parts of the infrastructure. Azure already has logic outside of the VM to abort hung provisioning. This changes lengthens the time out to 15 minutes. LP: #1717611