summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2021-04-09sysconfig: use BONDING_MODULE_OPTS on SUSE (#831)Jens Sandmann
Update sysconfig configuration to use BONDING_MODULES_OPTS instead of BONDING_OPTS when on a SUSE system. The sysconfig support requires use of BONDING_MODULE_OPTS whereas the initscript support that rhel uses requires BONDING_OPTS.
2021-04-07lp-to-git-users: adding B1Sandmann (#828)Jens Sandmann
2021-04-07bringup_static_routes: fix gateway check (#850)Petr Fedchenkov
When bringing up DHCP-provided static routes, we check for "0.0.0.0/0" to indicate an unspecified gateway. However, when parsing the static route in `parse_static_routes`, the gateway is never specified with a net length, so the "/0" will never happen. This change updates the gateway check to check only for "0.0.0.0".
2021-03-30add hamalq user (#860)hamalq
2021-03-30Add support to resize rootfs if using LVM (#721)Eduardo Otubo
This patch adds support to resize a single partition of a VM if it's using an LVM underneath. The patch detects if it's LVM if the given block device is a device mapper by its name (e.g. `/dev/dm-1`) and if it has slave devices under it on sysfs. After that syspath is updated to the real block device and growpart will be called to resize it (and automatically its Physical Volume). The Volume Group will be updated automatically and a final call to extend the rootfs to the remaining space available will be made. Using the same growpart configuration, the user can specify only one device to be resized when using LVM and growpart, otherwise cloud-init won't know which one should be resized and will fail. rhbz: #1810878 LP: #1799953 Signed-off-by: Eduardo Otubo <otubo@redhat.com> Signed-off-by: Scott Moser <smoser@brickies.net>
2021-03-29Fix mis-detecting network configuration in initramfs cmdline (#844)James Falcon
klibc initramfs in debian allows the 'iscsi_target_ip=' cmdline parameter to specify an iscsi device attachment. This can cause cloud-init to mis-detect the cmdline paramter as a networking config. LP: #1919188
2021-03-25tools/write-ssh-key-fingerprints: do not display empty header/footer (#817)dermotbradley
When output of SSH host keys and/or SSH fingerprints are disabled for all keys do not display headers and footers. Prevent risk of message text being interpreted as "logger" option by appending "--" to logger options. Correct syslog output that was tagged with "ec2" regardless of DataSource in use. Now use "cloud-init" tag instead. Various "shellcheck" corrections. Add testcase for disabled output of SSH host keys.
2021-03-25Azure helper: Ensure Azure http handler sleeps between retries (#842)Johnson Shi
Ensure that the Azure helper's http handler sleeps a fixed duration between retry failure attempts. The http handler will sleep a fixed duration between failed attempts regardless of whether the attempt failed due to (1) request timing out or (2) instant failure (no timeout). Due to certain platform issues, the http request to the Azure endpoint may instantly fail without reaching the http timeout duration. Without sleeping a fixed duration in between retry attempts, the http handler will loop through the max retry attempts quickly. This causes the communication between cloud-init and the Azure platform to be less resilient due to the short total duration if there is no sleep in between retries.
2021-03-24Fix chef apt source example (#826)timothegenzmer
key is a property of source1 and not sources
2021-03-19.travis.yml: generate an SSH key before running tests (#848)Daniel Watkins
2021-03-19write passwords only to serial console, lock down cloud-init-output.log (#847)Daniel Watkins
Prior to this commit, when a user specified configuration which would generate random passwords for users, cloud-init would cause those passwords to be written to the serial console by emitting them on stderr. In the default configuration, any stdout or stderr emitted by cloud-init is also written to `/var/log/cloud-init-output.log`. This file is world-readable, meaning that those randomly-generated passwords were available to be read by any user with access to the system. This presents an obvious security issue. This commit responds to this issue in two ways: * We address the direct issue by moving from writing the passwords to sys.stderr to writing them directly to /dev/console (via util.multi_log); this means that the passwords will never end up in cloud-init-output.log * To avoid future issues like this, we also modify the logging code so that any files created in a log sink subprocess will only be owner/group readable and, if it exists, will be owned by the adm group. This results in `/var/log/cloud-init-output.log` no longer being world-readable, meaning that if there are other parts of the codebase that are emitting sensitive data intended for the serial console, that data is no longer available to all users of the system. LP: #1918303
2021-03-19Fix apt default integration test (#845)James Falcon
The apt default test wasn't ported over from cloud-tests correctly. uri should be specified in the test, but it was not, so the test failed on openstack (and likely other platforms) because without a specified uri, the default uri will vary by platform. I separated this uri test out into a separate test function. Also add openstack specific test for apt configuration with no uri. Other platform-specific tests should be added here over time.
2021-03-18integration_tests: bump pycloudlib dependency (#846)Daniel Watkins
The latest pycloudlib now launches official Ubuntu cloud images for xenial, meaning that `lxc exec` no longer works against them. This commit includes handling for tests which are affected by this change; further details and reasoning in the included comment.
2021-03-16Fix stack trace if vendordata_raw contained an array (#837)eb3095
The implementation in existing datasources means that vendordata_raw is not "raw" as it ideally would be. Instead, actual values may include bytes, string or list. If the value was a list, then the attempt to persist that data to a file in '_store_rawdata' would raise a TypeError. The change is to encode with util.json_dumps (which is safe for binary data) before writing.
2021-03-15archlinux: Fix broken locale logic (#841)Kristian Klausen
The locale wasn't persisted correct nor set. LP: #1402406
2021-03-11Integration test for #783 (#832)James Falcon
Newer verisons of /etc/sudoers prefer @includedir over #includedir. Ensure we handle that properly and don't include an additional #includedir when one isn't warranted.
2021-03-11integration_tests: mount more paths IN_PLACE (#838)Daniel Watkins
This mounts the full directories that we install into systems over their corresponding paths within the system under test, getting us slightly closer to testing what a package would install.
2021-03-08Fix requiring device-number on EC2 derivatives (#836)James Falcon
#342 (70dbccbb) introduced the ability to determine route-metrics based on the `device-number` provided by the EC2 IMDS. Not all datasources that subclass EC2 will have this attribute, so allow the old behavior if `device-number` is not present. LP: #1917875
2021-03-08Remove the vi comment from the part-handler example (#835)James Falcon
2021-03-08net: exclude OVS internal interfaces in get_interfaces (#829)Daniel Watkins
`get_interfaces` is used to in two ways, broadly: firstly, to determine the available interfaces when converting cloud network configuration formats to cloud-init's network configuration formats; and, secondly, to ensure that any interfaces which are specified in network configuration are (a) available, and (b) named correctly. The first of these is unaffected by this commit, as no clouds support Open vSwitch configuration in their network configuration formats. For the second, we check that MAC addresses of physical devices are unique. In some OVS configurations, there are OVS-created devices which have duplicate MAC addresses, either with each other or with physical devices. As these interfaces are created by OVS, we can be confident that (a) they will be available when appropriate, and (b) that OVS will name them correctly. As such, this commit excludes any OVS-internal interfaces from the set of interfaces returned by `get_interfaces`. LP: #1912844
2021-03-03tox.ini: pass OS_* environment variables to integration tests (#830)Daniel Watkins
This allows source'd OpenStack credentials to be used for tox tests.
2021-03-03integration_tests: add OpenStack as a platform (#804)James Falcon
2021-03-03Add flexibility to IMDS api-version (#793)Thomas Stringer
Add flexibility to IMDS api-version by having both a desired IMDS api-version and a minimum api-version. The desired api-version will be used first, and if that fails it will fall back to the minimum api-version.
2021-03-03Fix the TestApt tests using apt-key on Xenial and Hirsute (#823)Paride Legovini
* Xenial issue The `apt-key finger` format changed since Xenial. Sample Xenial output: pub 4096R/991BC93C 2018-09-17 Key fingerprint = F6EC B376 2474 EDA9 D21B 7022 8719 20D1 991B Sample Focal output: pub rsa4096 2016-04-12 [SC] EB4C 1BFD 4F04 2F6D DDCC EC91 7721 F63B D38B 4796 What didn't change is the format of the key fingerprint, which should be enough to ensure that the right key is in place across all the supported releases. * Hirsute issue TestApt::test_ppa_source also fails on Hirsute because of a difference in how the PPA keys are added. On Focla this command: add-apt-repository ppa:simplestreams-dev/trunk install /etc/apt/trusted.gpg.d/simplestreams-dev_ubuntu_trunk.gpg, while on Hirsute the file is names simplestreams-dev-ubuntu-trunk.gpg. The filename is part of the `apt-key finger` output, and this the test fails. Only checking for the presence of the key fingerprint in apt-key also covers this case. LP: #1916629
2021-02-25doc: remove duplicate "it" from nocloud.rst (#825)V.I. Wood
2021-02-25archlinux: Use hostnamectl to set the transient hostname (#797)Kristian Klausen
hostname (inetutils) isn't installed per default on arch, so switch to hostnamectl which is installed per default (systemd).
2021-02-24cc_keys_to_console.py: Add documentation for recently added config key (#824)dermotbradley
PR #811 added a new config key, emit_keys_to_console, but didn't update the documentation for mention it.
2021-02-23Update cc_set_hostname documentation (#818)Toshi Aoyama
It is distro dependent whether hostname or fqdn is used
2021-02-23Release 21.1 (#820)Daniel Watkins
Bump the version in cloudinit/version.py to 21.1 and update ChangeLog. LP: #1916540
2021-02-22Azure: Support for VMs without ephemeral resource disks. (#800)Johnson Shi
Changes: * Only merge in default Azure cloud ephemeral disk configs during DataSourceAzure._get_data() if the ephemeral disk exists. * DataSourceAzure.address_ephemeral_resize() (which is invoked in DataSourceAzure.activate() should only set up the ephemeral disk if the disk exists. Azure VMs may or may not come with ephemeral resource disks depending on the VM SKU. For VM SKUs that come with ephemeral resource disks, the Azure platform guarantees that the ephemeral resource disk is attached to the VM before the VM is booted. For VM SKUs that do not come with ephemeral resource disks, cloud-init currently attempts to wait and set up a non-existent ephemeral resource disk, which wastes boot time. It also causes disk setup modules to fail (due to non-existent references to the ephemeral resource disk). udevadm settle is invoked by cloud-init very early in boot. udevadm settle is invoked very early, before DataSourceAzure's _get_data() and activate() methods. Within DataSourceAzure's _get_data() and activate() methods, the ephemeral resource disk path should exist if the VM SKU comes with an ephemeral resource disk. The ephemeral resource disk path should not exist if the VM SKU does not come with an ephemeral resource disk. LP: #1901011
2021-02-22cc_keys_to_console: add option to disable key emission (#811)Michael Hudson-Doyle
Specifically: ssh: emit_keys_to_console: false We also port the cc_keys_to_console cloud tests to the new integration testing framework, and add a test for this new option. LP: #1915460
2021-02-22integration_tests: introduce lxd_use_exec mark (#802)Daniel Watkins
pycloudlib has modified the way LXD executes tests (https://github.com/canonical/pycloudlib/pull/114): it will always use SSH to access them by default, instead of using `lxc exec`. This behaviour is transparent for them majority of cloud-init's integration tests, but some currently depend on using `lxc exec` to access instances with (intentionally) broken networking: obviously these are not accessible via SSH. pycloudlib retains support for switching an instance to use `lxc exec`. This commit introduces the `lxd_use_exec` mark, which tests can use to indicate to the integration testing framework that they should be so switched, and applies it to all applicable tests.
2021-02-19azure: case-insensitive UUID to avoid new IID during kernel upgrade (#798)Chad Smith
Kernel's newer than 4.15 present /sys/dmi/id/product_uuid as a lowercase value. Previously UUID was uppercase. Azure datasource reads the product_uuid directly as their platform's instance-id. This presents a problem if a kernel is either upgraded or downgraded across the 4.15 kernel version boundary because the case of the UUID will change, resulting in cloud-init seeing a "new" instance id and re-running all modules. Re-running cc_ssh in cloud-init deletes and regenerates ssh_host keys on a system which can cause concern on long-running instances that somethingnefarious has happened. Also add: - An integration test for this for Azure Bionic Ubuntu FIPS upgrading from a FIPS kernel with uppercase UUID to a lowercase UUID in linux-azure - A new pytest.mark.sru_next to collect all integration tests related to our next SRU LP: #1835584
2021-02-18stale.yml: don't ask submitters to reopen PRs (#816)Daniel Watkins
Because they don't have the permissions to do it. Instead, reword the message to ask people to ping mitechie, in line with the rest of the message.
2021-02-18integration_tests: fix use of SSH agent within tox (#815)Daniel Watkins
We need to allow the SSH_AUTH_SOCK environment variable through for paramiko to be able to find the agent.
2021-02-18integration_tests: add UPGRADE CloudInitSource (#812)Daniel Watkins
This allows out-of-date images to be brought up-to-date with the archive, so that tests written against the latest cloud-init release will pass.
2021-02-18integration_tests: use unique MAC addresses for tests (#813)Daniel Watkins
Using the same MAC address results in strange test behaviour if more than one such instance is up: traffic gets routed to an arbitrary interface with the given MAC address. This can happen if running tests in parallel, or on a system which retains test instances from previous runs. The introduction of tests/integration_tests/__init__.py means that pylint now checks the integration tests: this commit also addresses those failures.
2021-02-18Update .gitignore (#814)James Falcon
2021-02-16Port apt cloud_tests to integration tests (#808)James Falcon
2021-02-16integration_tests: fix test_gh626 on LXD VMs (#809)Daniel Watkins
Without a MAC address match clause, the test network configuration is not applied to the primary interface in LXD VMs (which is named enp*s* rather than eth0).
2021-02-16Fix attempting to decode binary data in test_seed_random_data test (#806)James Falcon
`test_seed_random_data.py` was failing on openstack as openstack provides additional binary seed data to the end of the specified file. The test has been changed to only read the ascii porition of seed file.
2021-02-09Remove wait argument from tests with session_cloud calls (#805)James Falcon
2021-02-08Datasource for UpCloud (#743)Antti Myyrä
New datasource utilizing UpCloud metadata API, including relevant unit tests and documentation.
2021-02-05test_gh668: fix failure on LXD VMs (#801)Daniel Watkins
In LXD containers, the default interface is named eth0. In VMs, it isn't; it's renamed by systemd (likely to enp5s0, but we can't rely on that). This means that, on VMs, the network configuration we specify for "eth0" doesn't match an interface in the system and so is not applied. This modifies the test to set a MAC address in a match clause in the network configuration and on the eth0 interface (which is the LXD name in both containers and VMs pre-rename): this ensures that the specified configuration applies in both cases.
2021-02-05openstack: read the dynamic metadata group vendor_data2.json (#777)Andrew Bogott
Add support for openstack's dynamic vendor data, which appears under openstack/latest/vendor_data2.json This adds vendor_data2 to all pathways; it should be a no-op for non-OpenStack providers. LP: #1841104
2021-01-29includedir in suoders can be prefixed by "arroba" (#783)Jordi Massaguer Pla
Since version 1.9.1, @includedir can be used in the sudoers files instead of #includedir: https://github.com/sudo-project/sudo/releases/tag/SUDO_1_9_1 Actually "@includedir" is the modern syntax, and "#includedir" the historic syntax. It has been considered that "#includedir" was too puzzling because it started with a "#" that otherwise denotes comments. This happens to be the default in SUSE Linux enterprise sudoer package, so cloudinit should take this into account. Otherwise, cloudinit was adding an extra #includedir, which was resulting on the files under /etc/sudoers.d being included twice, one by @includedir from the SUSE package, one by the @includedir from cloudinit. The consequence of this, was that if you were defining an Cmnd_Alias inside any of those files, this was being defined twice and creating an error when using sudo.
2021-01-28Merge upstream/20.4.1 into masterDaniel Watkins
upstream/20.4.1 was used to release the 20.4.1 hotfix. Merging it into master reconciles that fork in history, and integrates both the 20.4.1 changelog and tag into our main history.
2021-01-27[VMware] change default max wait time to 15s (#774)xiaofengw-vmware
If cloud-init is enabled on VMware platform, cloud-init will wait until its configuration file is ready and currently the max wait is 90 seconds by default. With our test, this configuration file should be ready within 1 second, so change it to 15 seconds for better performance. Also update the documentation about how to change the default value in cloud-init configuration file.
2021-01-26Revert integration test associated with reverted #586 (#784)James Falcon
2021-01-26Add jordimassaguerpla as contributor (#787)Jordi Massaguer Pla
Signed-off-by: Jordi Massaguer Pla <jmassaguerpla@suse.de>