summaryrefslogtreecommitdiff
path: root/cloudinit
AgeCommit message (Collapse)Author
2019-10-04get_interfaces: don't exclude bridge and bond membersDaniel Watkins
The change that introduced this issue was handling interfaces that are bonded in the kernel, in a way that doesn't present as "a bond" to userspace in the normal way. Both members of this "bond" will share a MAC address, so we filter one of them out to avoid incorrect MAC address collision warnings. Unfortunately, the matching condition was too broad, so that change also affected normal bonds and bridges. This change specifically excludes bonds and bridges from that determination, to address that regression. LP: #1846535
2019-10-01Add support for Arch Linux in render-cloudcfgConrad Hoffmann
 - Detect Arch Linux and set variant accordingly in `system_info()`  - Allow setting render-cloudcfg variant parameter to 'arch'  - Adjust some basic settings for Arch Linux in the cloud.cfg.tmpl The template might need some additional Arch-specific tweaks in the future, but at least for now the generated config works and contains the most relevant modules. Also: - Sort distro variant lists when adding Arch - Add debian to known variants in render-cloudcfg
2019-09-27util: json.dumps on python 2.7 will handle UnicodeDecodeError on binaryChad Smith
Since python 2.7 doesn't handle UnicodeDecodeErrors with the default handler LP: #1801364
2019-09-27debian/ubuntu: add missing word to netplan/ENI headerDaniel Watkins
Specifically, add in "reboot" to make it clear what people should expect when modifying the file. This also renames the variable to indicate it is used for netplan and ENI, not just ENI. LP: #1845669
2019-09-27ovf: do not generate random instance-id for IMC customization pathRyan Harper
Cloud-init will not operate properly if the instance-id value changes on each boot. This is the source of a number of behavioral bugs filed against cloud-init with OVF datasource. Instead, use a static instance-id value, iid-vmware-imc, similar to iid-dsovf.
2019-09-26sysconfig: only write resolv.conf if network_state has DNS valuesRyan Harper
If an OS image provided an /etc/resolv.conf file that was not empty cloud-init would read and re-write it with a cloud-init header even if no DNS network configuration was provided (e.g. DHCP only). This can cause problems for some network services which don't ignore cloud-init's header. LP: #1843634
2019-09-26sysconfig: use distro variant to check if availableRyan Harper
The sysconfig renderer used the distro name directly which mean some variants of distros were not considered supported. Fix this by using util.system_info()['variant'] instead. Fix the list of KNOWN_DISTROS value for redhat -> rhel. LP: #1843584
2019-09-25analyze/show: remove trailing space in outputDaniel Watkins
2019-09-25Add missing space in warning: "not avalid seed"Brian Candler
2019-09-18Add datasource for ZStack platform.Shixin Ruan
Zstack platform provides a AWS Ec2 metadata service, and identifies their platform to the guest by setting the 'chassis asset tag' to a string that ends with '.zstack.io'. LP: #1841181
2019-09-17net: add is_master check for filtering device listRyan Harper
Some network devices are transformed into a bond via kernel magic and do not have the 'bonding' sysfs attribute, but like a bond they have a duplicate MAC of other bond members. On Azure Advanced Networking SRIOV devices are auto bonded and will have the same MAC as the HyperV nic. We can detect this via the 'master' sysfs attribute in the device sysfs path and this patch adds this to the list of devices we ignore when enumerating device lists. LP: #1844191
2019-09-11Brightbox: restrict detection to require full domain match .brightbox.comScott Moser
The detection for brightbox in both ds-identify and in identify_brightbox would incorrectly match the domain 'bobrightbox', which is not a brightbox platform. The fix here is to restrict matching to '*.brightbox.com' rather than '*brightbox.com' Also, while here remove a url to bug 1661693 which added the knowledge of brightbox.
2019-09-11VMWware: add option into VMTools config to enable/disable custom script.Xiaofeng Wang
VMWware customization already has support to run a custom script during the VM customization. Adding this option allows a VM administrator to disable the execution of customization scripts. If set the script will not execute and the customization status is set to GUESTCUST_ERROR_SCRIPT_DISABLED.
2019-09-09net,Oracle: Add support for netfailover detectionRyan Harper
Add support for detecting netfailover[1] device 3-tuple in networking layer. In the Oracle datasource ensure that if a provided network config, either fallback or provided config includes a netfailover master to remove any MAC address value as this can break under 3-netdev as the other two devices have the same MAC. 1. https://www.kernel.org/doc/html/latest/networking/net_failover.html
2019-09-09atomic_helper: add DEBUG logging to write_fileDaniel Watkins
LP: #1843276
2019-08-28Oracle: Render secondary vnic IP and MTU values onlyRyan Harper
When rendering secondary vnic configuration from IMDS, only emit configuration for the IP and MTU values only. Add support to mutate either a v1 or a v2 network_config input.
2019-08-28exoscale: fix sysconfig cloud_config_modules overridesChad Smith
Make sure Exoscale supplements or overrides existing system config setting cloud_config_modules instead of replacing it with a one item list set-passords LP: #1841454
2019-08-22net/cmdline: refactor to allow multiple initramfs network config sourcesDaniel Watkins
This refactors read_initramfs_config to support multiple different types of initramfs network configuration. It introduces an InitramfsNetworkConfigSource abstract base class. There is currently a single sub-class, KlibcNetworkConfigSource, which contains the logic which previously was directly within read_initramfs_config.
2019-08-22ubuntu-drivers: call db_x_loadtemplatefile to accept NVIDIA EULAChad Smith
Emit a script allowing cloud-init to set linux/nvidia/latelink debconf selection to true. This avoids having to call debconf-set-selections and allows cloud-init to pre-confgure linux-restricted-modules to link NVIDIA drivers to the running kernel. Cloud-init loads this debconf template and sets the value to true in the debconf database by sourcing debconf's /usr/share/debconf/confmodule and uses db_x_loadtemplatefile to register cloud-init's setting for linux/nvidia/latelink. LP: #1840080
2019-08-19ubuntu-drivers: emit latelink=true debconf to accept nvidia eulaChad Smith
To accept NVIDIA EULA, cloud-init needs to emit latelink=true debconf setting to the linux-restricted-modules package to allow NVIDIA drivers to properly link to the running kernel. LP: #1840080
2019-08-19DataSourceOracle: prefer DS network config over initramfsDaniel Watkins
The Oracle platform provides networking configuration from two sources: * the primary interface configuration comes from the initramfs, because Oracle instance all iSCSI boot * secondary interface configuration comes from an IMDS accessed over HTTP As we need to combine these two sources of network configuration, the default "prefer initramfs config over data source config" behaviour isn't appropriate; we would never get the IMDS interfaces via that route. Instead, the Oracle data source has code to combine these two sources, so we prefer its network configuration over the initramfs configuration. (This is not appropriate default behaviour, because _in general_ data sources won't know how to merge initramfs-provided configuration into their provided configuration, so switching this order for all data sources would result in initramfs configuration being discarded on any data source that implements network_config.)
2019-08-16cloudinit/distros/parsers/sys_conf: add docstring to SysConfDaniel Watkins
2019-08-15pyflakes: remove unused variableJoshua Powers
2019-08-14Azure: Record boot timestamps, system information, and diagnostic eventsAnh Vo
Collect and record the following information through KVP:  + timestamps related to kernel initialization and systemd activation    of cloud-init services  + system information including cloud-init version, kernel version,    distro version, and python version  + diagnostic events for the most common provisioning error issues    such as empty dhcp lease, corrupted ovf-env.xml, etc. + increasing the log frequency of polling IMDS during reprovision.
2019-08-14DataSourceOracle: configure secondary NICs on Virtual MachinesDaniel Watkins
Oracle Cloud Infrastructure's Instance Metadata Service provides network configuration information for non-primary NICs. This commit introduces support, on Virtual Machines[0], for fetching that network metadata, converting it to v1 network-config[1] and combining it into the network configuration generated for the primary interface. By default, this behaviour is not enabled. Configuring the Oracle datasource to `configure_secondary_nics` enables it: datasource: Oracle: configure_secondary_nics: true Failures to fetch and generate secondary NIC configuration will log a warning, but otherwise will not affect boot. [0] The expected use of the IMDS-provided network configuration is substantially different on Bare Metal Machines, so support for that will be addressed separately. [1] This is v1 config, because cloudinit.net.cmdline generates v1 config and we need to integrate the secondary NICs into that configuration.
2019-08-14distros: fix confusing variable namesDaniel Watkins
Building the subp arguments for a `useradd` call in a variable named `adduser_cmd` is extremely confusing; let's not do that. (This also changes the snap and freebsd variables to something more apropos.)
2019-08-13azure/net: generate_fallback_nic emits network v2 config instead of v1Chad Smith
The function generate_fallback_config is used by Azure by default when not consuming IMDS configuration data. This function is also used by any datasource which does not implement it's own network config. This simple fallback configuration sets up dhcp on the most likely NIC. It will now emit network v2 instead of network v1. This is a step toward moving all components talking in v2 and allows us to avoid costly conversions between v1 and v2 for newer distributions which rely on netplan.
2019-08-09Add support for publishing host keys to GCE guest attributesRick Wright
This adds an empty publish_host_keys() method to the default datasource that is called by cc_ssh.py. This feature can be controlled by the 'ssh_publish_hostkeys' config option. It is enabled by default but can be disabled by setting 'enabled' to false. Also, a blacklist of key types is supported. In addition, this change implements ssh_publish_hostkeys() for the GCE datasource, attempting to write the hostkeys to the instance's guest attributes. Using these hostkeys for ssh connections is currently supported by the alpha version of Google's 'gcloud' command-line tool. (On Google Compute Engine, this feature will be enabled by setting the 'enable-guest-attributes' metadata key to 'true' for the project/instance that you would like to use this feature for. When connecting to the instance for the first time using 'gcloud compute ssh' the hostkeys will be read from the guest attributes for the instance and written to the user's local known_hosts file for Google Compute Engine instances.)
2019-08-08New data source for the Exoscale.com cloud platformChris Glass
- dsidentify switches to the new Exoscale datasource on matching DMI name - New Exoscale datasource added Signed-off-by: Mathieu Corbin <mathieu.corbin@exoscale.ch>
2019-08-07cc_set_passwords: rewrite documentationDaniel Watkins
What we had previously was inaccurate in a few respects. LP: #1838794
2019-07-26net/cmdline: split interfaces_by_mac and init network config determinationDaniel Watkins
Previously "cmdline" network configuration could be either user-specified network-config=... configuration data, or initramfs-provided configuration data. Before data sources could modify the order in which network config sources were considered, this conflation didn't matter (and, indeed, in the default data source configuration it will continue to not matter). However, it _is_ desirable for a data source to be able to specify that its network configuration should be preferred over the initramfs-provided network configuration but still allow explicit network-config=... configuration passed to the kernel cmdline to continue to override both of those sources. (This also modifies the Oracle data source to use read_initramfs_config directly, which is effectively what it was using read_kernel_cmdline_config for previously.)
2019-07-23stages: allow data sources to override network config source orderDaniel Watkins
Currently, if a platform provides any network configuration via the "cmdline" method (i.e. network-data=... on the kernel command line, ip=... on the kernel command line, or iBFT config via /run/net-*.conf), the value of the data source's network_config property is completely ignored. This means that on platforms that use iSCSI boot (such as Oracle Compute Infrastructure), there is no way for the data source to configure any network interfaces other than those that have already been configured by the initramfs. This change allows data sources to specify the order in which network configuration sources are considered. Data sources that opt to use this mechanism will be expected to consume the command line network data and integrate it themselves. (The generic merging of network configuration sources was considered, but we concluded that the single use case we have presently (a) didn't warrant the increased complexity, and (b) didn't give us a broad enough view to be sure that our generic implementation would be sufficiently generic. This change in no way precludes a merging strategy in future.)
2019-07-18Fix bug rendering MTU on bond or vlan when input was netplan.Scott Moser
If input to network_state.parse_net_config_data was netplan (v2 yaml) then the network state would lose the mtu information on bond or vlan. LP: #1836949
2019-07-17net: update net sequence, include wait on netdevs, opensuse netrules pathRyan Harper
On systems with many interfaces, processing udev events may take a while. Cloud-init expects devices included in a provided network-configuration to be present when attempting to configure them. This patch adds a step in net configuration where it will check for devices provided in the configuration and if not found, issue udevadm settle commands to wait for them to appear. Additionally, the default path for udev persistent network rules 70-persistent-net.rules may also be written to systems which include the 75-net-generator.rules. During boot, cloud-init and the generator may race and interleave values causing issues. OpenSUSE will now use a newer file, 85-persistent-net-cloud-init.rules which will take precedence over values created by 75-net-generator and avoid collisions on the same file. LP: #1817368
2019-07-17Release 19.2Ryan Harper
Bump the version in cloudinit/version.py to be 19.2 and update ChangeLog. LP: #1836921
2019-07-16net: add rfc3442 (classless static routes) to EphemeralDHCPRyan Harper
The EphemeralDHCP context manager did not parse or handle rfc3442 classless static routes which prevented reading datasource metadata in some clouds. This branch adds support for extracting the field from the leases output, parsing the format and then adding the required iproute2 ip commands to apply (and teardown) the static routes. LP: #1821102
2019-07-16Support netplan renderer in Arch LinuxConrad Hoffmann
Support is for now implemented in such a way that it will fall back to the old `_write_network()` if netplan is not available on the image.
2019-07-16VMWare: Trigger the post customization script via cc_scripts module.Xiaofeng Wang
cloud-init does not trigger reboots of a VM therefore adding custom scripts to rc.local does not execute the post scripts. This patch moves post-scripts into per-instance scripts dir and has cc_scripts module run the post-scripts. Also in this branch: - Remove the sh interpreter and execute the customization script directly. - Update the unit test. LP: #1833192
2019-07-15Cloud-init analyze module: Added ability to analyze boot events.Sam Gilson
This branch introduces a new command line feature for cloud-init. Currently, the cloud-init module has the capability to analyze events in cloud-init.log in three ways: 'show', 'blame', 'dump'. These changes add a fourth capability, called 'boot'. Running the command 'cloud-init analyze boot' will provide the user three timestamps. 1) Timestamp for when the kernel starts initializing. 2) Timestamp for when the kernel finishes its initialization. 3) Timestamp for when systemd activates cloud-init. This feature enables cloud-init users to analyze different boot phases. This would aid in debugging performance issues related to cloud-init startup or tracking regression.
2019-07-15Update debian eni network configuration location, retain Ubuntu settingJanos Lenart
On Debian, ifupdown uses `source-directory /etc/network/interfaces.d` (for new installs) to include files. https://salsa.debian.org/debian/ifupdown/blob/master/debian/postinst#L23 The current filename, 50-cloud-init.cfg, does not match against the RE that is used to scan the directory for configurations (ASCII upper- and lower-case letters, ASCII digits, ASCII underscores, and ASCII minus-hyphens): https://salsa.debian.org/debian/ifupdown/blob/master/interfaces.5.pre#L122 Of course many installations use `source /etc/network/interfaces.d/*`, but not all.
2019-07-15net: skip bond interfaces in get_interfacesStanislav Makar
bonds may inherit mac address from a physical interface LP: #1812857
2019-07-10Fix a couple of issues raised by a coverity scanDaniel Watkins
* cc_lxd: fix copy/paste error in debug logging * DataSourceCloudSigma: remove unreachable code * This unreachable code was introduced in a refactor (in 2015) which removed the need for an exception handler, but retained the logging from the exception handler as an unreachable fall-through.
2019-07-03Add missing dsname for Hetzner Cloud datasourceMarkus Schade
2019-06-25azure: add region and AZ properties from imds compute location metadataChad Smith
This allows cloud-init query region to show valid region data for Azure
2019-06-21sysconfig: support more bonding optionsPenghui Liao
Currently, only a few bonding parameters can be configured on sysconfig systems. This patch aims to support more parameters documented on the docs site.
2019-05-31Fix spelling error making 'an Ubuntu' consistent.Brian Murray
2019-05-29netplan: update netplan key mappings for gratuitous-arpRyan Harper
Previous versions of netplan included a misspelling for the bond parameter around gratuitous-arp. This has been fixed and released and cloud-init needs to accept both values. This branch fixes the key that will be rendered and transforms the previous misspelling when capturing network_state. LP: #1827238
2019-05-28freebsd: ability to grow root file systemGonéri Le Bouder
- UFS file system support - GPT partition table support - add support for newfs's -L parameter (label) - move freebsd specific test from Azure to freebsd
2019-05-24freebsd: NoCloud data source supportGonéri Le Bouder
blkid is a Linux-only command. With this patch, cloud-init uses another approach to find the data source on FreeBSD. LP: #1645824
2019-05-10Azure: Return static fallback address as if failed to find endpointJason Zions (MSFT)
The Azure data source helper attempts to use information in the dhcp lease to find the Wireserver endpoint (IP address). Under some unusual circumstances, those attempts will fail. This change uses a static address, known to be always correct in the Azure public and sovereign clouds, when the helper fails to locate a valid dhcp lease. This address is not guaranteed to be correct in Azure Stack environments; it's still best to use the information from the lease whenever possible.