summaryrefslogtreecommitdiff
path: root/cloudinit
AgeCommit message (Collapse)Author
2021-02-22Azure: Support for VMs without ephemeral resource disks. (#800)Johnson Shi
Changes: * Only merge in default Azure cloud ephemeral disk configs during DataSourceAzure._get_data() if the ephemeral disk exists. * DataSourceAzure.address_ephemeral_resize() (which is invoked in DataSourceAzure.activate() should only set up the ephemeral disk if the disk exists. Azure VMs may or may not come with ephemeral resource disks depending on the VM SKU. For VM SKUs that come with ephemeral resource disks, the Azure platform guarantees that the ephemeral resource disk is attached to the VM before the VM is booted. For VM SKUs that do not come with ephemeral resource disks, cloud-init currently attempts to wait and set up a non-existent ephemeral resource disk, which wastes boot time. It also causes disk setup modules to fail (due to non-existent references to the ephemeral resource disk). udevadm settle is invoked by cloud-init very early in boot. udevadm settle is invoked very early, before DataSourceAzure's _get_data() and activate() methods. Within DataSourceAzure's _get_data() and activate() methods, the ephemeral resource disk path should exist if the VM SKU comes with an ephemeral resource disk. The ephemeral resource disk path should not exist if the VM SKU does not come with an ephemeral resource disk. LP: #1901011
2021-02-22cc_keys_to_console: add option to disable key emission (#811)Michael Hudson-Doyle
Specifically: ssh: emit_keys_to_console: false We also port the cc_keys_to_console cloud tests to the new integration testing framework, and add a test for this new option. LP: #1915460
2021-02-19azure: case-insensitive UUID to avoid new IID during kernel upgrade (#798)Chad Smith
Kernel's newer than 4.15 present /sys/dmi/id/product_uuid as a lowercase value. Previously UUID was uppercase. Azure datasource reads the product_uuid directly as their platform's instance-id. This presents a problem if a kernel is either upgraded or downgraded across the 4.15 kernel version boundary because the case of the UUID will change, resulting in cloud-init seeing a "new" instance id and re-running all modules. Re-running cc_ssh in cloud-init deletes and regenerates ssh_host keys on a system which can cause concern on long-running instances that somethingnefarious has happened. Also add: - An integration test for this for Azure Bionic Ubuntu FIPS upgrading from a FIPS kernel with uppercase UUID to a lowercase UUID in linux-azure - A new pytest.mark.sru_next to collect all integration tests related to our next SRU LP: #1835584
2021-02-08Datasource for UpCloud (#743)Antti Myyrä
New datasource utilizing UpCloud metadata API, including relevant unit tests and documentation.
2021-02-05openstack: read the dynamic metadata group vendor_data2.json (#777)Andrew Bogott
Add support for openstack's dynamic vendor data, which appears under openstack/latest/vendor_data2.json This adds vendor_data2 to all pathways; it should be a no-op for non-OpenStack providers. LP: #1841104
2021-01-29includedir in suoders can be prefixed by "arroba" (#783)Jordi Massaguer Pla
Since version 1.9.1, @includedir can be used in the sudoers files instead of #includedir: https://github.com/sudo-project/sudo/releases/tag/SUDO_1_9_1 Actually "@includedir" is the modern syntax, and "#includedir" the historic syntax. It has been considered that "#includedir" was too puzzling because it started with a "#" that otherwise denotes comments. This happens to be the default in SUSE Linux enterprise sudoer package, so cloudinit should take this into account. Otherwise, cloudinit was adding an extra #includedir, which was resulting on the files under /etc/sudoers.d being included twice, one by @includedir from the SUSE package, one by the @includedir from cloudinit. The consequence of this, was that if you were defining an Cmnd_Alias inside any of those files, this was being defined twice and creating an error when using sudo.
2021-01-28Merge upstream/20.4.1 into masterDaniel Watkins
upstream/20.4.1 was used to release the 20.4.1 hotfix. Merging it into master reconciles that fork in history, and integrates both the 20.4.1 changelog and tag into our main history.
2021-01-27[VMware] change default max wait time to 15s (#774)xiaofengw-vmware
If cloud-init is enabled on VMware platform, cloud-init will wait until its configuration file is ready and currently the max wait is 90 seconds by default. With our test, this configuration file should be ready within 1 second, so change it to 15 seconds for better performance. Also update the documentation about how to change the default value in cloud-init configuration file.
2021-01-20Use proper spelling for Red Hat (#778)Dan Kenigsberg
The company name has two distinct words. Signed-off-by: Dan Kenigsberg <danken@redhat.com>
2021-01-19Revert "ssh_util: handle non-default AuthorizedKeysFile config (#586)" (#775)Daniel Watkins
This reverts commit b0e73814db4027dba0b7dc0282e295b7f653325c.
2021-01-15Release 20.4.1Daniel Watkins
Bump the version in cloudinit/version.py to 20.4.1 and update ChangeLog. LP: #1911680
2021-01-15Revert "ssh_util: handle non-default AuthorizedKeysFile config (#586)"Daniel Watkins
This reverts commit b0e73814db4027dba0b7dc0282e295b7f653325c.
2021-01-13[VMware] Support cloudinit raw data feature (#691)xiaofengw-vmware
This feature will modify VMware datasource to read from meta data and user data which are specified by VMware vSphere user. If meta data/user data are found in cloud-init configuration directory, datasource will parse the meta data/network and user data from the configuration file, otherwise it will continue to parse them from traditional customization configuration file as before. The supported meta data file is in json or yaml format.
2021-01-13net: Fix static routes to host in eni renderer (#668)Pavel Abalikhin
Route '-net' parameter is incompatible with /32 IPv4 addresses so we have to use '-host' in that case.
2021-01-13cc_seed_random: update documentation and fix integration test (#771)Daniel Watkins
The documentation did not mention that the given data may not be the exact string written: the cloud's random data may be added to it. Additionally, the documentation of the command key was incorrect. test_seed_random_data was updated to check that the given data is a prefix of the written data, to match cloud-init's expected (and, now, documented) behaviour. LP: #1911227
2021-01-12archlinux: fix package upgrade command handling (#768)Bao Trinh
pacman uses `-u` instead of `upgrade` to trigger a system upgrade, fix the command handling so this is properly accounted for. as is, the resulting command attempts to install a (non-existent) `upgrade` package Co-authored-by: Rick Harding <rharding@mitechie.com>
2021-01-11Fix regression with handling of IMDS ssh keys (#760)Thomas Stringer
With the changes for SSH public keys to be retrieved from IMDS as a first option, when a key is passed through not in the raw SSH public key format it causes an issue and the key is not added to the user's authorized_keys file. This PR will temporarily disable this behavior until a permanent fix is put in place.
2021-01-07net_convert: add some missing help text (#755)Daniel Watkins
2021-01-07Missing IPV6_AUTOCONF=no to render sysconfig dhcp6 stateful on RHEL (#753)Eduardo Otubo
IPV6_AUTOCONF needs to be set to 'no' on RHEL so NetworkManager can properly acquire ipv6 address. rhbz: #1859695 Signed-off-by: Eduardo Otubo <otubo@redhat.com>
2021-01-04Azure: Add telemetry for poll IMDS (#741)Johnson Shi
2020-12-17cc_ca_certs: add RHEL support (#633)cawamata
This refactors cc_ca_certs to support non-ca-certificates distros, and adds RHEL support.
2020-12-16Azure: only generate config for NICs with addresses (#709)Thomas Stringer
Prevent network interfaces without IP addresses from being added to the generated network configuration.
2020-12-10Drop unnecessary shebang from cmd/main.py (#722)Eduardo Otubo
Fedora build system's rpmlint is complaining that there is a file with a shebang but no executable flag set. No need to have shebang on this file, so drop it. Signed-off-by: Eduardo Otubo <otubo@redhat.com> Co-authored-by: Scott Moser <smoser@brickies.net>
2020-12-01Adding BOOTPROTO = dhcp to render sysconfig dhcp6 stateful on RHEL (#685)Eduardo Otubo
BOOTPROTO needs to be set to 'dhcp' on RHEL so NetworkManager can properly acquire ipv6 address. rhbz: #1859695 Signed-off-by: Eduardo Otubo <otubo@redhat.com> Co-authored-by: Daniel Watkins <oddbloke@ubuntu.com> Co-authored-by: Scott Moser <smoser@brickies.net>
2020-11-25cc_apt_configure: add riscv64 as a ports arch (#687)Dimitri John Ledkov
2020-11-24Release 20.4 (#686)20.4James Falcon
Bump the version in cloudinit/version.py to 20.4 and update ChangeLog. LP: #1905440
2020-11-23Ability to hot-attach NICs to preprovisioned VMs before reprovisioning (#613)aswinrajamannar
Adds the ability to run the Azure preprovisioned VMs as NIC-less and then hot-attach them when assigned for reprovision. The NIC on the preprovisioned VM is hot-detached as soon as it reports ready and goes into wait for one or more interfaces to be hot-attached. Once they are attached, cloud-init gets the expected number of NICs (in case there are more than one) that will be attached from IMDS and waits until all of them are attached. After all the NICs are attached, reprovision proceeds as usual.
2020-11-20Support configuring SSH host certificates. (#660)Jonathan Lung
Existing config writes keys to /etc/ssh after deleting files matching a glob that includes certificate files. Since sshd looks for certificates in the same directory as the keys, a host certificate must be placed in this directory. This update enables the certificate's contents to be specified along with the keys. Co-authored-by: jonathan lung <lungj@heresjono.com> Co-authored-by: jonathan lung <jlung@kepler.space>
2020-11-19cc_resizefs on FreeBSD: Fix _can_skip_ufs_resize (#655)Mina Galić
On FreeBSD, if a UFS has trim: (-t) or MAC multilabel: (-l) flag, resize FS fail, because the _can_skip_ufs_resize check gets tripped up by the missing options. This was reported at FreeBSD Bugzilla: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=250496 and as LP: #1901958 Rather than fixing the parser as in the patches proposed there (and attempted in #636) this pull-request rips out all of it, and simplifies the code. We now use `growfs -N` and check if that returns an error. If it returns the correct kind of error, we can skip the resize, because we either are at the correct size, or the filesystem in question is broken or not UFS. If it returns the wrong kind of error, we just re-raise it. LP: #1901958
2020-11-19DataSourceAzure: push dmesg log to KVP (#670)Anh Vo
Pushing dmesg log to KVP to help troubleshoot VM boot issues
2020-11-18DataSourceAzure: update password for defuser if exists (#671)Anh Vo
cc_set_password will only update the password for the default user if cfg['password'] is set. The existing code of datasource Azure will fail to update the default user's password because it does not set that metadata. If the default user doesn't exist in the image, the current code works fine because the password is set during user create and not in cc_set_password
2020-11-18Azure helper: Increase Azure Endpoint HTTP retries (#619)Johnson Shi
Increase Azure Endpoint HTTP retries to handle occasional platform network blips. Introduce a common method http_with_retries in the azure.py helper, which will serve as the common HTTP request handler for all HTTP requests with the Azure endpoint. This method has builtin retries and reporting diagnostics logic.
2020-11-18DataSourceAzure: send failure signal on Azure datasource failure (#594)Johnson Shi
On systems where the Azure datasource is a viable platform for crawling metadata, cloud-init occasionally encounters fatal irrecoverable errors during the crawling of the Azure datasource. When this happens, cloud-init crashes, and Azure VM provisioning would fail. However, instead of failing immediately, the user will continue seeing provisioning for a long time until it times out with "OS Provisioning Timed Out" message. In these situations, cloud-init should report failure to the Azure datasource endpoint indicating provisioning failure. The user will immediately see provisioning terminate, giving them a much better failure experience instead of pointlessly waiting for OS provisioning timeout.
2020-11-18test_persistence: simplify VersionIsPoppedFromState (#674)Daniel Watkins
2020-11-18cli: add --system param to allow validating system user-data on a machine (#575)Chad Smith
Allow root user to validate the userdata provided to the launched machine using `cloud-init devel schema --system`
2020-11-17test_persistence: add VersionIsPoppedFromState test (#673)Daniel Watkins
2020-11-17introduce an upgrade framework and related testing (#659)Daniel Watkins
This commit does the following: * introduces the `cloudinit.persistence` module, containing `CloudInitPickleMixin` which provides lightweight versioning of objects' pickled representations (and associated testing) * introduces a basic upgrade testing framework (in `cloudinit.tests.test_upgrade`) which unpickles pickles from previous versions of cloud-init (stored in `tests/data/old_pickles`) and tests invariants that the current cloud-init codebase expects * uses the versioning framework to address an upgrade issue where `Distro.networking` could get into an unexpected state, and uses the upgrade testing framework to confirm that the issue is addressed
2020-11-17add --no-tty option to gpg (#669)Till Riedel
Make sure that gpg works even if the instance has no /dev/tty. This has been observed on Debian. LP: #1813396
2020-11-13DataSourceOpenNebula: exclude SRANDOM from context output (#665)Daniel Watkins
This is a new builtin variable that appeared in Ubuntu in 5.1~rc2-1ubuntu1 and started causing daily build failures.
2020-11-10Fix stacktrace in DataSourceRbxCloud if no metadata disk is found (#632)Scott Moser
Largely speaking, ds-identify protects from this scenario being hit, but if DataSourceRbxCloud ran and there was no metadata disks found (LABEL=CLOUDMD), then it would stacktrace. The fix is just to clean up the get_md function a little bit, and the explicitly check for False as a return value in _get_data.
2020-11-09Make wakeonlan Network Config v2 setting actually work (#626)dermotbradley
Add code so that specifying "wakeonlan: true" actually results in relevant configuration entry appearing in /etc/network/interfaces, Netplan, and sysconfig for RHEL and OpenSuse. Add testcases for the above.
2020-11-06replace usage of dmidecode with kenv on FreeBSD (#621)Mina Galić
FreeBSD lets us read out kernel parameters with kenv(1), a user-space utility that's shipped in "base" We can use it in place of dmidecode(8), thus removing the dependency on sysutils/dmidecode, and the restrictions to i386 and x86_64 architectures that this utility imposes on FreeBSD. Co-authored-by: Scott Moser <smoser@brickies.net>
2020-11-04azure: enable pushing the log to KVP from the last pushed byte (#614)Moustafa Moustafa
This allows the cloud-init log to be pushed multiple times during boot, with the latest lines being pushed each time.
2020-11-03Make some language improvements in growpart documentation (#649)Shane Frasier
* Fix awkward English in sentence * Add the missing word "the" * Fix misspelling * Add @jsf9k as a contributor Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-11-03Fix not sourcing default 50-cloud-init ENI file on Debian (#598)WebSpider
* Include both Ubuntu-style cfg file, and Debian-style directory in ENI * Add WebSpider as contributor
2020-11-02remove unnecessary reboot from gpart resize (#646)Mina Galić
a reboot after `gpart resize` hasn't been necessary since ca FreeBSD 9.2 Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-11-02cloudinit: move dmi functions out of util (#622)Scott Moser
This just separates the reading of dmi values into its own file. Some things of note: * left import of util in dmi.py only for 'is_container' It'd be good if is_container was not in util. * just the use of 'util.is_x86' to dmi.py * open() is used directly rather than load_file.
2020-11-02util: fix mounting of vfat on *BSD (#637)Mina Galić
Fix mounting of vfat filesystems by normalizing the different names for vfat to "msdos" which works across BSDs.
2020-10-30Correct documentation and testcase data for some user-data YAML (#618)dermotbradley
For cc_users_groups the user setting "expiredate" must be quoted in order for the relevant flag and value to be then passed to the useradd command. It its vaiue is not quoted then it is treated as Python type datetime.date and in `cloudinit/distros/__init__.py` the below "is it a string" condition fails and so no "--expiredate" parameter is passed to useradd and therefore it has no effect: ``` if key in useradd_opts and val and isinstance(val, str): useradd_cmd.extend([useradd_opts[key], val]) ``` For cc_users_groups, the user setting "inactive" does not actually disable accounts, the useradd "--inactive" option actually defines the number of days after password expiry that users can still login. So I have changed the docs to show it taking a quoted value of days (which works with the current code) rather than a boolean value. The quotes are necessary, like expiredate above, so that the value is also passed to the useradd command. For cc_power_state_change.py the "delay" setting value needs to have quotes around it as otherwise its leading plus sign will be stripped off.
2020-10-30Hetzner: Fix instance_id / SMBIOS serial comparison (#640)Markus Schade
Fixes erroneous string/int comparison introduced in 1431c8a metadata['instance-id'] is an integer but the value read from smbios is a string. The comparision would cause TypeError.