summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2019-01-30Resolve flake8 comparison and pycodestyle over-ident issuesParide Legovini
Fixes: - flake8: use ==/!= to compare str, bytes, and int literals - pycodestyle: E117 over-indented
2019-01-28opennebula: also exclude epochseconds from changed environment varsChad Smith
In addition to EPOCHREALTIME there is also an EPOCHSECONDS environment variable that OpenNebula needs to exclude as it is expected to change. This commit supplements the other exclusion in commit d1a2fe7307e9cf2251d1f9a666c12d71d3f522d6. Without this fix, unittests will intermittently fail if parse_shell_config is run across a timing boundary where the EPOCHSECONDS changes mid-test. LP: #1813641
2019-01-28systemd: Render generator from template to account for system differences.Robert Schweikert
The systemd generator used had a hard coded path for the location target file to create. This path does not apply to all distributions. Make the generator and template to have the path set during build time.
2019-01-28sysconfig: On SUSE, use STARTMODE instead of ONBOOTRobert Schweikert
ONBOOT is not recognized on openSUSE and SUSE Linux Enterprise, add the STARTMODE setting LP: #1799540
2019-01-28flake8: use ==/!= to compare str, bytes, and int literalsParide Legovini
2019-01-26opennebula: exclude EPOCHREALTIME as known bash env variable with a deltaChad Smith
This branch is needed to allow cloud-init to sbuild on Ubuntu Disco. OpenNebula:parse_shell_config tries to do a comparison of bash environment values, excluding expected environment variables which are known to change. Bash on Ubuntu Disco surfaces a new EPOCHREALTIME environment variable which wasn't in previous bash environments, this var needs to be ignored by parse_shell_config too. LP: #1813383
2019-01-25tox: fix disco httpretty dependencies for py37Chad Smith
LP: #1813361
2019-01-25run-container: uncomment baseurl in yum.repos.d/*.repo when using a proxyParide Legovini
When using a proxy it is often useful to know in advance which mirrors are to be contacted, so a whitelist can be set up. This is not easy when using the yum.conf(5) mirrorlist option, as the retrieved list of mirrors may change. The repository definition may also specify a canonical mirror with the 'baseurl' option; this option is often commented out by default to favor the usage of worldwide mirrors. This patch uncomments 'baseurl' when an http_proxy is being used, so the canonical mirror is used *in addition to* the mirrors retrieved from the mirrorlist.
2019-01-25lxd: install zfs-linux instead of zfs meta packageJohnson Shi
When using the LXD module cloud-init will attempt to install ZFS if it does not exist on the target system. However instead of installing the `zfsutils-linux` package it attempts to install `zfs` resulting in an error. Ubuntu Xenial (16.04) has zfs meta package, but Bionic (18.04) does not. Use the specific base package instead of zfs meta. Co-authored-by: Michael Skalka <michael.skalka@canonical.com> LP: #1799779
2019-01-23net/sysconfig: do not write a resolv.conf file with only the header.Robert Schweikert
Writing the file with no dns information may prevent distro tools from writing a resolv.conf file with dns information obtained from a dhcp server.
2019-01-18net: Make sysconfig renderer compatible with Network Manager.Eduardo Otubo
The 'sysconfig' renderer is activated if, and only if, there's ifup and ifdown commands present in its search dictonary or the network-scripts configuration files are found. This patch adds a check for Network- Manager configuration file as well. This solution is based on the use of the plugin 'ifcfg-rh' present in Network-Manager and is designed to support Fedora 29 or other distributions that also replaced network-scripts by Network-Manager.
2019-01-15cc_set_passwords: Fix regex when parsing hashed passwordsMarlin Cremers
Correct invalid regex to match hashes starting with the following: - $1, $2a, $2y, $5 or $6 LP: #1811446
2019-01-15net: Wait for dhclient to daemonize before reading lease fileJason Zions
cloud-init uses dhclient to fetch the DHCP lease so it can extract DHCP options. dhclient creates the leasefile, then writes to it; simply waiting for the leasefile to appear creates a race between dhclient and cloud-init. Instead, wait for dhclient to be parented by init. At that point, we know it has written to the leasefile, so it's safe to copy the file and kill the process. cloud-init creates a temporary directory in which to execute dhclient, and deletes that directory after it has killed the process. If cloud-init abandons waiting for dhclient to daemonize, it will still attempt to delete the temporary directory, but will not report an exception should that attempt fail. LP: #1794399
2019-01-15[Azure] Increase retries when talking to Wireserver during metadata walkJason Zions
Testing startup of large numbers of VMs (of varying distros) in Azure shows that 3 retries results in a small percentage of failed VMs. Increasing that by a few dramatically decreases the occurrence of provisioning timeout errors. The initial choice of "3 retries" was uninformed by heavy testing. Also, the alternate provisioning mechanism for Azure (waagent) retries the Wireserver crawl without limit. 10 retries seems a more reasonable choice.
2019-01-08Add documentation on adding a datasource.Scott Moser
This adds documentation intended for a developer on how to add a new datasource to cloud-init.
2019-01-08doc: clean up some datasource documentation.Scott Moser
The change to datasources.rst here is obvious typo fix. The change to azure is to reduce the two 'Customization' sections to a single and clean up some other duplicate text.
2018-12-20ds-identify: fix wrong variable name in ovf_vmware_transport_guestinfo.Scott Moser
ovf_vmware_transport_guestinfo is not currently tested. It used '$1' instead of '$out' when checking for xml content in the output of vmware-rpctool.
2018-12-20Scaleway: Support ssh keys provided inside an instance tag.PORTE Loïc
The change here will utilize ssh keys found inside an instance's tag. The tag value must start with 'AUTHORIZED_KEY'.
2018-12-20OVF: simplify expected return values of transport functions.Scott Moser
Transport functions (transport_iso9660 and transport_vmware_guestinfo) would return a tuple of 3 values, but only the first was ever used outside of test. The other values (device and filename) were just ignored. This just simplifies the transport functions to now return content (in string format) or None indicating that the transport was not found.
2018-12-20Vmware: Add support for the com.vmware.guestInfo OVF transport.Scott Moser
This adds support for reading OVF information over the 'com.vmware.guestInfo' tranport. The current implementation requires vmware-rpctool be installed in the system. LP: #1807466
2018-12-18HACKING.rst: change contact info to Josh PowersScott Moser
In the Hacking doc, change the contact information for the contributors agreement to reference Josh Powers rather than Scott Moser.
2018-12-14Update to pylint 2.2.2.Scott Moser
The tip-pylint tox target correctly reported the invalid use of string formatting. The change here is to: a.) Fix the error that was caught. b.) move to pylint 2.2.2 for the default 'pylint' target.
2018-12-13Release 18.5Ryan Harper
Bump the version in cloudinit/version.py to be 18.5 and update ChangeLog. LP: #1808380
2018-12-11tests: add Disco releaseJoshua Powers
2018-12-11net: render 'metric' values in per-subnet routesRyan Harper
It is possible to have a metric value in a per-subnet route. This is currently missing in all renderers. Update each renderer to emit the correct metric value from the config. LP: #1805871
2018-12-06write_files: add support for appending to files.James Baxter
Add 'append: true' to write_files entries to append 'content' to file specified by 'path' key. This modifies the file open mode to append.
2018-12-04config: On ubuntu select cloud archive mirrors for armel, armhf, arm64.Scott Moser
Infrastructure is now set up for Ubuntu to handle Amazon instances hitting a ports archive at: - http://%(ec2_region)s.ec2.ports.ubuntu.com/ubuntu-ports/ And additionally, generic mirrors at *.clouds.ports.ubuntu.com/ubuntu-ports The change here will utilize those mirrors for the arm64, armel and armhf arches. We've decided to limit the auto-selection of those mirrors to arm, where we know a use case. That way new instances of ppc64el or other arches will not select them. Such a behavior change could be problematic for a user in a firewalled environment. LP: #1805854
2018-12-03dhclient-hook: cleanups, tests and fix a bug on 'down' event.Scott Moser
I noticed a bug in dhclient_hook on the 'down' event, using 'is' operator rather than '==' (if self.net_action is 'down'). This refactors/simplifies the code a bit for easier testing and adds tests. The reason for the rename of 'action' to 'event' is to just be internally consistent. The word and Namespace 'action' is used by cloud-init main, so it was not really usable here. Also adds a main which can easily be debugged with: CI_DHCP_HOOK_DATA_D=./my.d python -m cloudinit.dhclient_hook up eth0
2018-12-03NoCloud: Allow top level 'network' key in network-config.Scott Moser
NoCloud's 'network-config' file was originally expected to contain network configuration without the top level 'network' key. This was because the file was named 'network-config' so specifying 'network' seemed redundant. However, JuJu is currently providing a top level 'network' config when it tries to disable networking ({"network": {"config": "disabled"}). Other users have also been surprised/confused by the fact that a network config in /etc/cloud/cloud.cfg.d/network.cfg differed from what was expected in 'network-config'. LP: #1798117
2018-12-03ovf: Fix ovf network config generation gateway/routesRyan Harper
Move routes under the nic's subnet rather than use top-level ("global") route config ensuring all net renderers will provide the configured route. Also updated cloudinit/cmd/devel/net_convert.py:  - Add input type 'vmware-imc' for OVF customization config files  - Fix bug when output-type was netplan which invoked netplan   generate/apply and attempted to write to   /etc/netplan/50-cloud-init.yaml instead of joining with the   output directory. LP: #1806103
2018-11-29azure: detect vnet migration via netlink media change eventTamilmani Manoharan
Replace Azure pre-provision polling on IMDS with a blocking call which watches for netlink link state change messages. The media change event happens when a pre-provisioned VM has been activated and is connected to the users virtual network and cloud-init can then resume operation to complete image instantiation.
2018-11-29Azure: fix copy/paste error in error handling when reading azure ovf.Adam DePue
Check the appropriate variables based on code review. Correcting what seems to be a copy/paste mistake for the error handling from a few lines above.
2018-11-28tests: fix incorrect order of mocks in test_handle_zfs_root.Scott Moser
The order of parameters to test_handle_zfs_root did not match the order of the mocks applied. Thanks to Jason Zions for pointing this out.
2018-11-28doc: Change dns_nameserver property to dns_nameservers.Tomer Cohen
According to the examples in the page, v1 network config DNS should be defined using the dns_nameservers. The singular dns_nameserver is undefined.
2018-11-27OVF: identify label iso9660 filesystems with label 'OVF ENV'.Scott Moser
When deploying an OVA, at least some versions of vmware attach a cdrom with an ISO9660 filesystem label of 'OVF ENV'. This was seen on Vmware vCenter Server, 6.0.0, 2776510. In order to accomplish this we had to change the content of the DI_ISO9660_DEVS variable to be comma delimited rather than space delimited.
2018-11-27logs: collect-logs ignore instance-data-sensitive.json on non-root userChad Smith
Since /run/cloud-init/instance-data-sensitive.json is root read-only, ignore this file if non-root user runs collect-logs. If --include-userdata is provided on the command line, exit in error if non-root user attempts this operation. Lastly, update the __main__ to exit based on return value of main. LP: #1805201
2018-11-26net: Ephemeral*Network: add connectivity check via URLChad Smith
We add a new Optional parameter: connectivity_url This is used in __enter__ to verify if a connection already exists. If it does exist, no operations are performed.
2018-11-15azure: _poll_imds only retry on 404. Fail on TimeoutChad Smith
Upon URL timeout, _poll_imds is expected to re-dhcp to get updated IP configuration. We don't want to indefinitely retry because the instance likely has invalid IP configuration. LP: #1803598
2018-11-14resizefs: Prefix discovered devpath with '/dev/' when path does not existIgor Galić
In some environments, like FreeBSD, gpart can return the device basename instead of the full path. If this discovered devpath does not exist and is missing the '/dev/' prefix, add that prefix in an attempt to find the device.
2018-11-13azure: retry imds polling on requests.TimeoutChad Smith
There is an infrequent race when the booting instance can hit the IMDS service before it is fully available. This results in a requests.ConnectTimeout being raised. Azure's retry_callback logic now retries on either 404s or Timeouts. LP:1800223
2018-11-12azure: Accept variation in error msg from mount for ntfs volumesJason Zions
If Azure detects an ntfs filesystem type during mount attempt, it should still report the resource device as reformattable. There are slight differences in error message format on RedHat and SuSE. This patch simplifies the expected error match to work on both distributions. LP: #1799338
2018-11-12azure: fix regression introduced when persisting ephemeral dhcp leaseasakkurr
In commitish 9073951 azure datasource tried to leverage stale DHCP information obtained from EphemeralDHCPv4 context manager to report updated provisioning status to the fabric earlier in the boot process. Unfortunately the stale ephemeral network configuration had already been torn down in preparation to bring up IMDS network config so the report attempt failed on timeout. This branch introduces obtain_lease and clean_network public methods on EphemeralDHCPv4 to allow for setup and teardown of ephemeral network configuration without using a context manager. Azure datasource now uses this to persist ephemeral network configuration across multiple contexts during provisioning to avoid multiple DHCP roundtrips.
2018-11-08azure: add udev rules to create cloud-init Gen2 disk name symlinksChad Smith
Cloud-init delivers udev rules on Azure to create the following symlinks: - /dev/disk/cloud/azure_root - /dev/disk/cloud/azure_root-part# - /dev/disk/cloud/azure_resource - /dev/disk/cloud/azure_resource-part# Cloud-init cc_disk_setup expects presence of these dev links in order to setup the mounted ephemeral disks. Gen1 instances udev rules match based only a DEVICE_ID attribute that no longer exists on Gen2 instances. Supplement existing Gen1 rules with matches on specitic SCSI target/lun path 0:0:0 and 0:0:1 and generate links for azure_root and azure_resource respectively. LP: #1797480
2018-11-08tests: ec2 mock missing httpretty user-data and instance-identity routesChad Smith
2018-11-01azure: remove /etc/netplan/90-hotplug-azure.yaml when net from IMDSChad Smith
There was a typo in the seeded filename s/azure-hotplug/hotplug-azure/.
2018-10-31azure: report ready to fabric after reprovision and reduce loggingasakkurr
When reusing a preprovisioned VM, report ready to Azure fabric as soon as we get the reprovision data and the goal state so that we are not delayed by the cloud-init stage switch, saving 2-3 seconds. Also reduce logging when polling IMDS for reprovision data. LP: #1799594
2018-10-30query: better error when missing read permission on instance-dataChad Smith
Emit a permissions error instead of "Missing instance-data.json" when non-root user doesn't have read-permission on /run/cloud-init/instance-data.json
2018-10-26instance-data: fallback to instance-data.json if sensitive is absent.Chad Smith
On cloud-init upgrade path from 18.3 to 18.4 cloud-init changed how instance-data is written. Cloud-init changes instance-data.json from root read-only to redacted world-readable content, and provided a separate unredacted instance-data-sensitive.json which is read-only root. Since instance-data is only rewritten from cache on reboot, the query and render tools needed fallback to use the 'old' instance-data.json if the new sensitive file isn't yet present. This avoids error messages from tools about an absebt /run/instance-data-sensitive.json file. LP: #1798189
2018-10-25docs: remove colon from network v1 config example.Tomer Cohen
The docs for network v1 config contained a errant ':'. Simply drop it.
2018-10-25Add cloud-id binary to packages for SUSEJason Zions