summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2019-03-21tox: Update testenv for openSUSE Leap to 15.0Thomas Bechtold
Use the requirements for the openSUSE Leap 15.0 release.
2019-03-21net: Fix ipv6 static routes when using eni rendererRaphael Glon
When rendering ipv6 static routes in eni format the post-up/pre down commands were not correct for ipv6. LP: #1818669
2019-03-19Add ubuntu_drivers config moduleDaniel Watkins
The ubuntu_drivers config module enables usage of the 'ubuntu-drivers' command. At this point it only serves as a way of installing NVIDIA drivers for general purpose graphics processing unit (GPGPU) functionality. Also, a small usability improvement to get_cfg_by_path to allow it to take a string for the key path "toplevel/second/mykey" in addition to the original: ("toplevel", "second", "mykey")
2019-03-18doc: Refresh Azure walinuxagent docsDaniel Watkins
- Remove outdated waagent.conf recommendations - Recommend using Provisioning.UseCloudInit - Reorganise sections so walinuxagent recommendations are easier to find
2019-03-18tox: bump pylint version to latest (2.3.1)Daniel Watkins
The previous version was emitting errors due to an incompatibility with one of its dependencies. (We could have pinned the dependency instead, but staying current on pylint is a worthy goal in and of itself.)
2019-03-14DataSource: move update_events from a class to an instance attributeDaniel Watkins
Currently, DataSourceAzure updates self.update_events in __init__. As update_events is a class attribute on DataSource, this updates it for all instances of classes derived from DataSource including those for other clouds. This means that if DataSourceAzure is even instantiated, its behaviour is applied to whichever data source ends up being used for boot. To address this, update_events is moved from a class attribute to an instance attribute (that is therefore populated at instantiation time). This retains the defaults for all DataSource sub-class instances, but avoids them being able to mutate the state in instances of other DataSource sub-classes. update_events is only ever referenced on an instance of DataSource (or a sub-class); no code relies on it being a class attribute. (In fact, it's only used within methods on DataSource or its sub-classes, so it doesn't even _need_ to remain public, though I think it's appropriate for it to be public.) DataSourceScaleway is also updated to move update_events from a class attribute to an instance attribute, as the class attribute would now be masked by the DataSource instance attribute. LP: #1819913
2019-03-12net/sysconfig: Handle default route setup for dhcp configured NICsRobert Schweikert
When the network configuration has a default route configured and another network device that is configured with dhcp, SUSE sysconfig output should not accept the default route provided by the dhcp server. LP: #1812117
2019-03-12DataSourceEc2: update RELEASE_BLOCKER to be more accurateDaniel Watkins
Our previous understanding of the upgrade issue was incomplete; it turns out the only change we need is the one now outlined.
2019-03-08cloud-init-per: POSIX sh does not support string subst, use sedRyan Harper
cloud-init-per is run via /bin/sh which requires POSIX shell compliance and does not implement string substitution like bash. Replace these calls with use of sed. LP: #1819222
2019-03-06Support locking user with usermod if passwd is not available.Scott Moser
In some cases, the 'passwd' command might not be available, but 'usermod' might be. In debian systems both are provided by the 'passwd' package. In Redhat/Centos passwd comes from 'passwd' package while 'usermod' comes from `shadow-utils` This should just support either one with no real cost other than the check.
2019-03-06Example for Microsoft Azure data disk added.Anton Olifir
2019-03-04clean: correctly determine the path for excluding seed directoryDaniel Watkins
Previously, init.paths.cloud_dir has a trailing slash, which meant that "/var/lib/cloud//seed" was being compared to "/var/lib/cloud/seed" and (of course), never matching. In this commit, switch to using os.path.join to avoid this case (and update the tests to catch it in future). LP: #1818571
2019-03-04helpers/openstack: Treat unknown link types as physicalDaniel Watkins
Some deployments of OpenStack expose link types to the guest which cloud-init doesn't recognise. These will almost always be physical, so we can operate more robustly if we assume that they are (whilst warning the user that we're seeing something unexpected). LP: #1639263
2019-03-04drop Python 2.6 support and our NIH version detectionDaniel Watkins
- Remove the last few places that use `if PY26` - Replace our Python version detection logic with six's (which we were already using in most places)
2019-03-04tip-pylint: Fix assignment-from-return-none errorsRyan Harper
pylint now complains about assignment of None from a return of a function call. This does not account for subclassing so we resolve this issue by removing the assignment in the unittest.
2019-03-04net: append type:dhcp[46] only if dhcp[46] is True in v2 netconfigKurt Stieger
When providing netplan configuration to cloud-init, the internal network state would enable DHCP if the 'dhcp' key was present in the source config. In netplan, dhcp[46] is a boolean and the value of the boolean should control whether DHCP is enabled rather than the presence of the key. This issue leaded to inconsistant sysconfig/network-scripts on fedora. 'BOOTPROTO' was always 'dhcp', even if the address config was static. After this change a dhcp subnet is added only if the 'dhcp' setting in source cfg dict is True. LP: #1818032
2019-02-27cc_apt_pipelining: stop disabling pipelining by defaultDaniel Watkins
This was introduced due to Ubuntu using S3 mirrors, and S3 having a buggy pipelining implementation. Those Ubuntu mirrors are no longer in production and, furthremore, apt has also grown the ability to handle servers with broken pipelining. As such, we can stop disabling pipelining, which should result in improved apt download speeds. LP: #1794982
2019-02-26tests: fix some slow tests and some leaking stateDaniel Watkins
In test_ds_identify, don't mutate otherwise-static test data. When running tests in a random order, this was causing failures due to breaking preconditions for other tests. In tests/helpers, reset logging level in tearDown. Some of the CLI tests set the level of the root logger in a way that isn't correctly reset. For test_poll_imds_re_dhcp_on_timeout and test_dhcp_discovery_run_in_sandbox_warns_invalid_pid, mock out time.sleep; this saves ~11 seconds (or ~40% of previous test time!).
2019-02-26util: don't determine string_types ourselvesDaniel Watkins
six already provides this for us, and we're already paying the cost to determine it there; no need to do it twice.
2019-02-26cc_rsyslog: Escape possible nested setDaniel Watkins
Under Python 3.7, we are seeing `FutureWarning: Possible nested set at position 23`; escaping this bracket causes that warning to disappear. LP: #1816967
2019-02-25Enable encrypted_data_bag_secret support for ChefEric Williams
Encrypted data bags require a secrets file to be present to decrypt, and the location of the file must be configured the Chef client configuration file, client.rb. This update enables cloud-init's chef module to update that setting in client.rb. LP: #1817082
2019-02-22azure: Filter list of ssh keys pulled from fabricJason Zions (MSFT)
The Azure data source is expected to expose a list of ssh keys for the user-to-be-provisioned in the crawled metadata. When configured to use the __builtin__ agent this list is built by the WALinuxAgentShim. The shim retrieves the full set of certificates and public keys exposed to the VM from the wireserver, extracts any ssh keys it can, and returns that list. This fix reduces that list of ssh keys to just the ones whose fingerprints appear in the "administrative user" section of the ovf-env.xml file. The Azure control plane exposes other ssh keys to the VM for other reasons, but those should not be added to the authorized_keys file for the provisioned user.
2019-02-22doc: update merging doc with fixes and some additional details/examplesRyan Harper
Update config merging documentation with cloud-config syntax fix. Add an example showing how to merge two files with runcmd.
2019-02-22tests: integration test failure summary to use traceback if empty errorChad Smith
When integration tests verification fails, the object returned contains has 'error' and 'traceback' keys. Each key can contain empty strings. If the simplified 'error' message is empty, fallback and use the more verbose full 'traceback' text in the failure summary.
2019-02-18This is to fix https://bugs.launchpad.net/cloud-init/+bug/1812676Vitaly Kuznetsov
2019-02-14EC2: Rewrite network config on AWS Classic instances every bootGuilherme G. Piccoli
AWS EC2 instances' network come in 2 basic flavors: Classic and VPC (Virtual Private Cloud). The former has an interesting behavior of having its MAC address changed whenever the instance is stopped/restarted. This behavior is not observed in VPC instances. In Ubuntu 18.04 (Bionic) the network "management" changed from ENI-style (etc/network/interfaces) to netplan, and when using netplan we observe the following block present in /etc/netplan/50-cloud-init.yaml: match: macaddress: aa:bb:cc:dd:ee:ff Jani Ollikainen noticed in Launchpad bug #1802073 that the EC2 Classic instances were booting without network access in Bionic after stop/restart procedure, due to their MAC address change behavior. It was narrowed down to the netplan MAC match block, that kept the old MAC address after stopping and restarting an instance, since the network configuration writing happens by default only once in EC2 instances, in the first boot. This patch changes the network configuration write to every boot in EC2 Classic instances, by checking against the "vpc-id" metadata information provided only in the VPC instances - if we don't have this metadata value, cloud-init will rewrite the network configuration file in every boot. This was tested in an EC2 Classic instance and proved to fix the issue; unit tests were also added for the new method is_classic_instance(). LP: #1802073 Reported-by: Jani Ollikainen <jani.ollikainen@ik.fi> Suggested-by: Ryan Harper <ryan.harper@canonical.com> Co-developed-by: Chad Smith <chad.smith@canonical.com> Signed-off-by: Guilherme G. Piccoli <gpiccoli@canonical.com>
2019-02-08netinfo: Adjust ifconfig output parsing for FreeBSD ipv6 entriesRyan Harper
FreeBSD ifconfig output for ipv6 addrs doesn't find scopeid values when present in the output and the pformat rendering assumes that an ipv6 address will have a 'scope6' entry in the netdev info dictionary. This patch finds the scopeid value, which is not always inside <>, and in some cases v6 addrs don't have a scopeid value in the output, so when rendering the table, allow scope6 value to be replaced with the empty value. LP: #1779672
2019-02-07netplan: Don't render yaml aliases when dumping netplanRyan Harper
Cloud-init rendered netplan with duplicate aliases if a network config included "global" nameserver/search values. Netplan uses can read yaml files which do use aliaes but cloud-init did not render a single yaml dictionary, instead it combined yaml sections into a single document which sometimes resulted in duplicate aliases being present. This branch introduces a yaml SafeDumper class which can set the 'ignore_aliases' attribute. This is not enabled by default but callers to util.yaml_dumps can pass a boolean to toggle this. The netplan render uses noalias=True and the resulting yaml output does not contain any aliases. LP: #1815051
2019-02-07add PyCharm IDE .idea/ path to .gitignoreDominic Schlegel
2019-02-06correct grammar issue in instance metadata documentationDominic Schlegel
LP: #1802188
2019-01-31clean: cloud-init clean should not trace when run from within cloud_dirChad Smith
Avoid traceback when cloud-init clean is run from within /var/lib/cloud/ deleted dirs. LP: #1795508
2019-01-30Resolve flake8 comparison and pycodestyle over-ident issuesParide Legovini
Fixes: - flake8: use ==/!= to compare str, bytes, and int literals - pycodestyle: E117 over-indented
2019-01-28opennebula: also exclude epochseconds from changed environment varsChad Smith
In addition to EPOCHREALTIME there is also an EPOCHSECONDS environment variable that OpenNebula needs to exclude as it is expected to change. This commit supplements the other exclusion in commit d1a2fe7307e9cf2251d1f9a666c12d71d3f522d6. Without this fix, unittests will intermittently fail if parse_shell_config is run across a timing boundary where the EPOCHSECONDS changes mid-test. LP: #1813641
2019-01-28systemd: Render generator from template to account for system differences.Robert Schweikert
The systemd generator used had a hard coded path for the location target file to create. This path does not apply to all distributions. Make the generator and template to have the path set during build time.
2019-01-28sysconfig: On SUSE, use STARTMODE instead of ONBOOTRobert Schweikert
ONBOOT is not recognized on openSUSE and SUSE Linux Enterprise, add the STARTMODE setting LP: #1799540
2019-01-28flake8: use ==/!= to compare str, bytes, and int literalsParide Legovini
2019-01-26opennebula: exclude EPOCHREALTIME as known bash env variable with a deltaChad Smith
This branch is needed to allow cloud-init to sbuild on Ubuntu Disco. OpenNebula:parse_shell_config tries to do a comparison of bash environment values, excluding expected environment variables which are known to change. Bash on Ubuntu Disco surfaces a new EPOCHREALTIME environment variable which wasn't in previous bash environments, this var needs to be ignored by parse_shell_config too. LP: #1813383
2019-01-25tox: fix disco httpretty dependencies for py37Chad Smith
LP: #1813361
2019-01-25run-container: uncomment baseurl in yum.repos.d/*.repo when using a proxyParide Legovini
When using a proxy it is often useful to know in advance which mirrors are to be contacted, so a whitelist can be set up. This is not easy when using the yum.conf(5) mirrorlist option, as the retrieved list of mirrors may change. The repository definition may also specify a canonical mirror with the 'baseurl' option; this option is often commented out by default to favor the usage of worldwide mirrors. This patch uncomments 'baseurl' when an http_proxy is being used, so the canonical mirror is used *in addition to* the mirrors retrieved from the mirrorlist.
2019-01-25lxd: install zfs-linux instead of zfs meta packageJohnson Shi
When using the LXD module cloud-init will attempt to install ZFS if it does not exist on the target system. However instead of installing the `zfsutils-linux` package it attempts to install `zfs` resulting in an error. Ubuntu Xenial (16.04) has zfs meta package, but Bionic (18.04) does not. Use the specific base package instead of zfs meta. Co-authored-by: Michael Skalka <michael.skalka@canonical.com> LP: #1799779
2019-01-23net/sysconfig: do not write a resolv.conf file with only the header.Robert Schweikert
Writing the file with no dns information may prevent distro tools from writing a resolv.conf file with dns information obtained from a dhcp server.
2019-01-18net: Make sysconfig renderer compatible with Network Manager.Eduardo Otubo
The 'sysconfig' renderer is activated if, and only if, there's ifup and ifdown commands present in its search dictonary or the network-scripts configuration files are found. This patch adds a check for Network- Manager configuration file as well. This solution is based on the use of the plugin 'ifcfg-rh' present in Network-Manager and is designed to support Fedora 29 or other distributions that also replaced network-scripts by Network-Manager.
2019-01-15cc_set_passwords: Fix regex when parsing hashed passwordsMarlin Cremers
Correct invalid regex to match hashes starting with the following: - $1, $2a, $2y, $5 or $6 LP: #1811446
2019-01-15net: Wait for dhclient to daemonize before reading lease fileJason Zions
cloud-init uses dhclient to fetch the DHCP lease so it can extract DHCP options. dhclient creates the leasefile, then writes to it; simply waiting for the leasefile to appear creates a race between dhclient and cloud-init. Instead, wait for dhclient to be parented by init. At that point, we know it has written to the leasefile, so it's safe to copy the file and kill the process. cloud-init creates a temporary directory in which to execute dhclient, and deletes that directory after it has killed the process. If cloud-init abandons waiting for dhclient to daemonize, it will still attempt to delete the temporary directory, but will not report an exception should that attempt fail. LP: #1794399
2019-01-15[Azure] Increase retries when talking to Wireserver during metadata walkJason Zions
Testing startup of large numbers of VMs (of varying distros) in Azure shows that 3 retries results in a small percentage of failed VMs. Increasing that by a few dramatically decreases the occurrence of provisioning timeout errors. The initial choice of "3 retries" was uninformed by heavy testing. Also, the alternate provisioning mechanism for Azure (waagent) retries the Wireserver crawl without limit. 10 retries seems a more reasonable choice.
2019-01-08Add documentation on adding a datasource.Scott Moser
This adds documentation intended for a developer on how to add a new datasource to cloud-init.
2019-01-08doc: clean up some datasource documentation.Scott Moser
The change to datasources.rst here is obvious typo fix. The change to azure is to reduce the two 'Customization' sections to a single and clean up some other duplicate text.
2018-12-20ds-identify: fix wrong variable name in ovf_vmware_transport_guestinfo.Scott Moser
ovf_vmware_transport_guestinfo is not currently tested. It used '$1' instead of '$out' when checking for xml content in the output of vmware-rpctool.
2018-12-20Scaleway: Support ssh keys provided inside an instance tag.PORTE Loïc
The change here will utilize ssh keys found inside an instance's tag. The tag value must start with 'AUTHORIZED_KEY'.
2018-12-20OVF: simplify expected return values of transport functions.Scott Moser
Transport functions (transport_iso9660 and transport_vmware_guestinfo) would return a tuple of 3 values, but only the first was ever used outside of test. The other values (device and filename) were just ignored. This just simplifies the transport functions to now return content (in string format) or None indicating that the transport was not found.