summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2019-08-13azure/net: generate_fallback_nic emits network v2 config instead of v1Chad Smith
The function generate_fallback_config is used by Azure by default when not consuming IMDS configuration data. This function is also used by any datasource which does not implement it's own network config. This simple fallback configuration sets up dhcp on the most likely NIC. It will now emit network v2 instead of network v1. This is a step toward moving all components talking in v2 and allows us to avoid costly conversions between v1 and v2 for newer distributions which rely on netplan.
2019-08-09Add support for publishing host keys to GCE guest attributesRick Wright
This adds an empty publish_host_keys() method to the default datasource that is called by cc_ssh.py. This feature can be controlled by the 'ssh_publish_hostkeys' config option. It is enabled by default but can be disabled by setting 'enabled' to false. Also, a blacklist of key types is supported. In addition, this change implements ssh_publish_hostkeys() for the GCE datasource, attempting to write the hostkeys to the instance's guest attributes. Using these hostkeys for ssh connections is currently supported by the alpha version of Google's 'gcloud' command-line tool. (On Google Compute Engine, this feature will be enabled by setting the 'enable-guest-attributes' metadata key to 'true' for the project/instance that you would like to use this feature for. When connecting to the instance for the first time using 'gcloud compute ssh' the hostkeys will be read from the guest attributes for the instance and written to the user's local known_hosts file for Google Compute Engine instances.)
2019-08-08New data source for the Exoscale.com cloud platformChris Glass
- dsidentify switches to the new Exoscale datasource on matching DMI name - New Exoscale datasource added Signed-off-by: Mathieu Corbin <mathieu.corbin@exoscale.ch>
2019-08-08doc: remove intersphinx extensionDaniel Watkins
We don't use it in our doc builds, and if sphinx.pocoo.org is down then it causes doc build issues.
2019-08-07cc_set_passwords: rewrite documentationDaniel Watkins
What we had previously was inaccurate in a few respects. LP: #1838794
2019-07-26net/cmdline: split interfaces_by_mac and init network config determinationDaniel Watkins
Previously "cmdline" network configuration could be either user-specified network-config=... configuration data, or initramfs-provided configuration data. Before data sources could modify the order in which network config sources were considered, this conflation didn't matter (and, indeed, in the default data source configuration it will continue to not matter). However, it _is_ desirable for a data source to be able to specify that its network configuration should be preferred over the initramfs-provided network configuration but still allow explicit network-config=... configuration passed to the kernel cmdline to continue to override both of those sources. (This also modifies the Oracle data source to use read_initramfs_config directly, which is effectively what it was using read_kernel_cmdline_config for previously.)
2019-07-23stages: allow data sources to override network config source orderDaniel Watkins
Currently, if a platform provides any network configuration via the "cmdline" method (i.e. network-data=... on the kernel command line, ip=... on the kernel command line, or iBFT config via /run/net-*.conf), the value of the data source's network_config property is completely ignored. This means that on platforms that use iSCSI boot (such as Oracle Compute Infrastructure), there is no way for the data source to configure any network interfaces other than those that have already been configured by the initramfs. This change allows data sources to specify the order in which network configuration sources are considered. Data sources that opt to use this mechanism will be expected to consume the command line network data and integrate it themselves. (The generic merging of network configuration sources was considered, but we concluded that the single use case we have presently (a) didn't warrant the increased complexity, and (b) didn't give us a broad enough view to be sure that our generic implementation would be sufficiently generic. This change in no way precludes a merging strategy in future.)
2019-07-18cloud_tests: updates and fixesRyan Harper
- Update paramiko and cryptography module versions (2.4.2) to address issues with algo and deprecation warnings. - Modify ssh keypair generation to work with updated paramiko - tools/xkvm sync with newer version from curtin - Update NoCloudKvm instance.py to work with updated xkvm - pass -name to instance, useful for debugging on shared host - Add cache_mode platform config; default to cache=none,aio=native - Switch to yaml.safe_load() in platforms.py
2019-07-18Fix bug rendering MTU on bond or vlan when input was netplan.Scott Moser
If input to network_state.parse_net_config_data was netplan (v2 yaml) then the network state would lose the mtu information on bond or vlan. LP: #1836949
2019-07-17net: update net sequence, include wait on netdevs, opensuse netrules pathRyan Harper
On systems with many interfaces, processing udev events may take a while. Cloud-init expects devices included in a provided network-configuration to be present when attempting to configure them. This patch adds a step in net configuration where it will check for devices provided in the configuration and if not found, issue udevadm settle commands to wait for them to appear. Additionally, the default path for udev persistent network rules 70-persistent-net.rules may also be written to systems which include the 75-net-generator.rules. During boot, cloud-init and the generator may race and interleave values causing issues. OpenSUSE will now use a newer file, 85-persistent-net-cloud-init.rules which will take precedence over values created by 75-net-generator and avoid collisions on the same file. LP: #1817368
2019-07-17Release 19.2Ryan Harper
Bump the version in cloudinit/version.py to be 19.2 and update ChangeLog. LP: #1836921
2019-07-16net: add rfc3442 (classless static routes) to EphemeralDHCPRyan Harper
The EphemeralDHCP context manager did not parse or handle rfc3442 classless static routes which prevented reading datasource metadata in some clouds. This branch adds support for extracting the field from the leases output, parsing the format and then adding the required iproute2 ip commands to apply (and teardown) the static routes. LP: #1821102
2019-07-16templates/ntp.conf.debian.tmpl: fix missing newline for poolsRyan Harper
The debian ntp.conf template did not contain a newline for the comment used to mark the rendered ntp pools configured. This resulted in an invalid line: '# poolspool 0.int.pool.ntp.org iburst' rather than: '# pools  pool 0.int.pool.ntp.org iburst' This patch fixes the template and updates the unittest to verify that the rendered templates puts servers and pools at the beginning of a line. LP: #1836598
2019-07-16Support netplan renderer in Arch LinuxConrad Hoffmann
Support is for now implemented in such a way that it will fall back to the old `_write_network()` if netplan is not available on the image.
2019-07-16Fix typo in publicly viewable documentation.David Medberry
2019-07-16Add a cdrom size checker for OVF ds to ds-identifyPengpeng Sun
With a large size ISO file attached to iso dev, ds-identify might grep it entirely if iso dev is ISO9660, it takes very long time to start OS. Resolve this by: - Adding a checker to read the ISO size (from sysfs). If the size of the ISO filesystem is > 10MiB then the ISO will be ignored (logged as oversized). - Move the ovf vmware guest customization checker to be ahead of cdrom ovf checker, so no need check the ISO size if vmware guest customization is enabled. LP: #1806701
2019-07-16VMWare: Trigger the post customization script via cc_scripts module.Xiaofeng Wang
cloud-init does not trigger reboots of a VM therefore adding custom scripts to rc.local does not execute the post scripts. This patch moves post-scripts into per-instance scripts dir and has cc_scripts module run the post-scripts. Also in this branch: - Remove the sh interpreter and execute the customization script directly. - Update the unit test. LP: #1833192
2019-07-15Cloud-init analyze module: Added ability to analyze boot events.Sam Gilson
This branch introduces a new command line feature for cloud-init. Currently, the cloud-init module has the capability to analyze events in cloud-init.log in three ways: 'show', 'blame', 'dump'. These changes add a fourth capability, called 'boot'. Running the command 'cloud-init analyze boot' will provide the user three timestamps. 1) Timestamp for when the kernel starts initializing. 2) Timestamp for when the kernel finishes its initialization. 3) Timestamp for when systemd activates cloud-init. This feature enables cloud-init users to analyze different boot phases. This would aid in debugging performance issues related to cloud-init startup or tracking regression.
2019-07-15Update debian eni network configuration location, retain Ubuntu settingJanos Lenart
On Debian, ifupdown uses `source-directory /etc/network/interfaces.d` (for new installs) to include files. https://salsa.debian.org/debian/ifupdown/blob/master/debian/postinst#L23 The current filename, 50-cloud-init.cfg, does not match against the RE that is used to scan the directory for configurations (ASCII upper- and lower-case letters, ASCII digits, ASCII underscores, and ASCII minus-hyphens): https://salsa.debian.org/debian/ifupdown/blob/master/interfaces.5.pre#L122 Of course many installations use `source /etc/network/interfaces.d/*`, but not all.
2019-07-15net: skip bond interfaces in get_interfacesStanislav Makar
bonds may inherit mac address from a physical interface LP: #1812857
2019-07-10Fix a couple of issues raised by a coverity scanDaniel Watkins
* cc_lxd: fix copy/paste error in debug logging * DataSourceCloudSigma: remove unreachable code * This unreachable code was introduced in a refactor (in 2015) which removed the need for an exception handler, but retained the logging from the exception handler as an unreachable fall-through.
2019-07-03Add missing dsname for Hetzner Cloud datasourceMarkus Schade
2019-06-25doc: indicate that netplan is default in Ubuntu nowDaniel Watkins
2019-06-25azure: add region and AZ properties from imds compute location metadataChad Smith
This allows cloud-init query region to show valid region data for Azure
2019-06-21sysconfig: support more bonding optionsPenghui Liao
Currently, only a few bonding parameters can be configured on sysconfig systems. This patch aims to support more parameters documented on the docs site.
2019-06-19cloud-init-generator: use libexec path to ds-identify on redhat systemsRyan Harper
Update the template to use libexec prefix path to ds-identify on redhat systems. LP: #1833264
2019-06-17tools/build-on-freebsd: update to python3Gonéri Le Bouder
- use python3 by default - ability to use any Python version through the PYTHON env-var - indent with 4 spaces - use 'set -eux' - remove trailing whitespace - drop the cheetah dep, Jinja2 is enough
2019-06-03Allow identification of OpenStack by Asset TagMark T. Voelker
When OpenStack is deployed on some hypervisors (such as VMware vSphere), cloud-init doesn't detect that it needs to probe the metadata service because the DMI product name field can't be set to a field that is recognized by cloud-init. However, the asset tag field can be set via flavor extra specs or image metadata. A similar approach is already used to identify Open Telekom Cloud. This patch allows cloud init to recognize "OpenStack Nova" or "OpenStack Compute" in the asset tag field as an indication that the instance being configured is running on an OpenStack platform. LP: #1669875
2019-05-31Fix spelling error making 'an Ubuntu' consistent.Brian Murray
2019-05-30run-container: centos: comment out the repo mirrorlistParide Legovini
In this way only the 'baseurl' mirror is used, which is easier to allow through firewalls and proxies.
2019-05-29netplan: update netplan key mappings for gratuitous-arpRyan Harper
Previous versions of netplan included a misspelling for the bond parameter around gratuitous-arp. This has been fixed and released and cloud-init needs to accept both values. This branch fixes the key that will be rendered and transforms the previous misspelling when capturing network_state. LP: #1827238
2019-05-28freebsd: fix the name of cloudcfg VARIANTGonéri Le Bouder
config/cloud.cfg.tmpl uses 'freebsd', not 'bsd' to identify FreeBSD.
2019-05-28freebsd: ability to grow root file systemGonéri Le Bouder
- UFS file system support - GPT partition table support - add support for newfs's -L parameter (label) - move freebsd specific test from Azure to freebsd
2019-05-24freebsd: NoCloud data source supportGonéri Le Bouder
blkid is a Linux-only command. With this patch, cloud-init uses another approach to find the data source on FreeBSD. LP: #1645824
2019-05-10Azure: Return static fallback address as if failed to find endpointJason Zions (MSFT)
The Azure data source helper attempts to use information in the dhcp lease to find the Wireserver endpoint (IP address). Under some unusual circumstances, those attempts will fail. This change uses a static address, known to be always correct in the Azure public and sovereign clouds, when the helper fails to locate a valid dhcp lease. This address is not guaranteed to be correct in Azure Stack environments; it's still best to use the information from the lease whenever possible.
2019-05-09release 19.1Chad Smith
Bump the version on cloudinit/version.py to be 19.1 and update ChangeLog LP: #1828479
2019-05-09freebsd: add chpasswd pkg in the imageGonéri Le Bouder
cc_set_passwords.py depends on chpasswd binary.
2019-05-09tests: add Eoan releaseParide Legovini
2019-05-08cc_mounts: check if mount -a on no-change fstab pathJason Zions (MSFT)
Under some circumstances, cc_disk_setup may reformat volumes which already appear in /etc/fstab (e.g. Azure ephemeral drive is reformatted from NTFS to ext4 after service-heal). Normally, cc_mounts only calls mount -a if it altered /etc/fstab. With this change cc_mounts will read /proc/mounts and verify if configured mounts are already mounted and if not raise flag to request a mount -a. This handles the case where no changes to fstab occur but a mount -a is required due to change in underlying device which prevented the .mount unit from running until after disk was reformatted. LP: #1825596
2019-05-08replace remaining occurrences of LOG.warnDaniel Watkins
2019-05-08DataSourceAzure: Adjust timeout for polling IMDSAnh Vo
If the IMDS primary server is not available, falling back to the secondary server takes about 1s. The net result is that the expected E2E time is slightly more than 1s. This change increases the timeout to 2s to prevent the infinite loop of timeouts.
2019-04-29Azure: Changes to the Hyper-V KVP ReporterAnh Vo
 + Truncate KVP Pool file to prevent stale entries from being processed by the Hyper-V KVP reporter.  + Drop filtering of KVPs as it is no longer needed.  + Batch appending of existing KVP entries.
2019-04-27git tests: no longer show warning about safe yaml.Scott Moser
Currently on 18.04, running tox -e py27 will spew errors like: .tests/unittests/test_net.py:2649: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. The change here just uses cloud-init's yaml, which does safeloading by default.
2019-04-23tools/read-version: handle errorsChad Miller
When the cloned branch was not the canonical upstream and tags were not available, tox would fail because tools/read-version would fail, and tragically never print the advice that is in tools/read-version about how to fix it. This changes tools/read-version to catch the exception that is elsewhere explicitly thrown and treat that too as an error it can handle.
2019-04-22net/sysconfig: only indicate available on known sysconfig distrosRyan Harper
Restrict the sysconfig renderer availabily to known distros. Ubuntu/Debian systems may include network-manager but they do not have support for reading sysconfig network output; that is enabled via a Network-Manager plugin: ifcfg-rh which is not available in Ubuntu/Debian. LP: #1819994
2019-04-19packages: update rpm specs for new bash completion pathDaniel Watkins
LP: #1825444
2019-04-18test_azure: mock util.SeLinuxGuard where neededJason Zions (MSFT)
Mock util.SeLinuxGuard to do nothing within tests that mock functions used by the guard, when those mocks confuse the guard. This has no impact when executing unit tests on systems which do not enable selinux (e.g. Ubuntu). LP: #1825253
2019-04-18setup.py: install bash completion script in new locationDaniel Watkins
Per lintian, this is the path at which bash completion scripts should now be installed.
2019-04-18mount_cb: do not pass sync and rw options to mountGonéri Le Bouder
On FreeBSD, mount_cd9660 does not accept the sync option that is enabled by default. In addition, the sync is only useful with the `rw` mode. However the `rw` mode was never used. This patch removes the `rw` and `sync` parameter of `mount_cb` to simplify the code base and resolve the FreeBSD issue. LP: #1645824
2019-04-17cc_apt_configure: fix typo in apt documentationDominic Schlegel