summaryrefslogtreecommitdiff
path: root/cloudinit
AgeCommit message (Collapse)Author
2021-07-12ssh-util: allow cloudinit to merge all ssh keys into a custom user file, ↵Emanuele Giuseppe Esposito
defined in AuthorizedKeysFile (#937) This patch aims to fix LP1911680, by analyzing the files provided in sshd_config and merge all keys into an user-specific file. Also introduces additional tests to cover this specific case. The file is picked by analyzing the path given in AuthorizedKeysFile. If it points inside the current user folder (path is /home/user/*), it means it is an user-specific file, so we can copy all user-keys there. If it contains a %u or %h, it means that there will be a specific authorized_keys file for each user, so we can copy all user-keys there. If no path points to an user-specific file, for example when only /etc/ssh/authorized_keys is given, default to ~/.ssh/authorized_keys. Note that if there are more than a single user-specific file, the last one will be picked. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Co-authored-by: James Falcon <therealfalcon@gmail.com> LP: #1911680 RHBZ:1862967
2021-07-08VMware: new "allow_raw_data" switch (#939)xiaofengw-vmware
Add a new switch allow_raw_data to control raw data feature, update the documentation. Fix bugs about max_wait.
2021-07-01freebsd/net_v1 format: read MTU from root (#930)Gonéri Le Bouder
We read the MTU from the subnet entries. With the v1 format, the MTU can be set at the root level of the interface entry in the `config` section. Limitation, we won't set the MTU if the interface use DHCP. This would require a bit of refactoring. Also simplify/clarify how we pass the target variable in `cloudinit.net.bsd`. See: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256309 Reported-by: Andrey Fesenko
2021-07-01Add new network activators to bring up interfaces (#919)James Falcon
Currently _bring_up_interfaces() is a no-op for any distro using renderers. We need to be able to support bringing up a single interfaces, a list of interfaces, and all interfaces. This should be independent of the renderers, as the network config is often generated independent of the mechanism used to apply it. Additionally, I included a refactor to remove "_supported_write_network_config". We had a confusing call chain of apply_network_config->_write_network_config->_supported_write_network_config. The last two have been combined.
2021-07-01- Detect a Python version change and clear the cache (#857)Robert Schweikert
summary: Clear cache when a Python version change is detected When a distribution gets updated it is possible that the Python version changes. Python makes no guarantee that pickle is consistent across versions as such we need to purge the cache and start over. Co-authored-by: James Falcon <therealfalcon@gmail.com>
2021-06-28Removed distro specific network code from Photon (#929)sshedi
Minor fixes in networkd renderer & fixed corresponding tests Removed datasource_list for Photon from cloud.cfg.tmpl & added a comment in cloud.cfg.tmpl about not to use multiline array for datasource_list. Signed-off-by: Shreenidhi Shedi <sshedi@vmware.com>
2021-06-18Add support for VMware PhotonOS (#909)sshedi
Also added a new (currently experimental) systemd-networkd renderer, and includes a small refactor to cc_resolv_conf.py to support the resolved.conf used by systemd-resolved.
2021-06-17Fix DNS in NetworkState (SC-133) (#923)James Falcon
v1 network config currently has no concept of interface-specific DNS, which is required for certain renderers. To fix this, added an optional 'interface' key on the v1 nameserver definition. If specified, it makes the DNS settings specific to the interface. Otherwise, it will be defined as global DNS as it always has. Additionally, DNS for v2 wasn't being recognized correctly. For DNS defined on a particular interface, these settings now also go into the global DNS settings as they were intended.
2021-06-17Fix the spelling of "DigitalOcean" (#924)Mark Mercado
The name "DigitalOcean" doesn't have a space in it; it's a single compound word written in Pascal case (upper camel case).
2021-06-16Small Doc Update for ReportEventStack and Test (#920)Mike Russell
- small document update for ReportEventStack explaining post_files parameter - small unit test for test_reporting demonstrating the close of an event with optional post_files list
2021-06-15Replace deprecated collections.Iterable with abc replacement (#922)James Falcon
LP: #1932048
2021-06-14add DragonFlyBSD support (#904)Gonéri Le Bouder
- Mostly based on FreeBSD, the main exception is that `find_devs_with_on_freebsd` does not work. - Since we cannot get the CDROM or the partition labels, `find_devs_with_on_dragonflybsd()` has a more naive approach and returns all the block devices.
2021-06-10Use instance-data-sensitive.json in jinja templates (SC-117) (#917)James Falcon
instance-data.json redacts sensitive data for non-root users. Since user data is consumed as root, we should be consuming the non-redacted data instead. LP: #1931392
2021-06-02- Create the log file with 640 permissions (#858)Robert Schweikert
Security scanners are often simple minded and complain on arbitrary settings such as file permissions. For /var/log/* having world read is one of these cases.
2021-06-01Allow braces to appear in dhclient output (#911)eb3095
dhclient output that contains brackets for pxe variables will break the dhclient parsing regex line. This fix retains the current functionality while fixing this particular issue.
2021-05-25openbsd/net: flush the route table on net restart (#908)Gonéri Le Bouder
Ensure we've got a clean environment before we restart the network. In some cases, the `sh /etc/netstart` is not enough to restart the network. A previous default route remains in the route table and as a result the network is broken. Also `sh /netstart` does not kill `dhclient`. The problen happens for instance with OVH OpenStack SBG3.
2021-05-25Add Rocky Linux support to cloud-init (#906)Louis Abel
Rocky Linux is a RHEL-compatible distribution so all changes that have been made should be trivial.
2021-05-18Added support for importing keys via primary/security mirror clauses (#882)Paul Goins
Presently, mirror keys cannot be associated with primary/security mirrors. Unfortunately, this prevents use of Landscape-managed package mirrors as the mirror key for the Landscape-hosted repository cannot be provided. This patch allows the same key-related fields usable on "sources" entries to be used on the "primary" and "security" entries as well. LP: #1925395
2021-05-14BSD: static network, set the mtu (#894)Gonéri Le Bouder
In the case of a static network, we now set the MTU according to the meta-data.
2021-05-14Fix unit tests breaking from new httpretty version (#903)James Falcon
httpretty now logs all requests by default which gets mixed up with our logging tests. Also we were incorrectly setting a logging level to 'None', which now also causes issues with the new httpretty version. See https://github.com/gabrielfalcao/HTTPretty/pull/419
2021-05-13Allow user control over update events (#834)James Falcon
Control is currently limited to boot events, though this should allow us to more easily incorporate HOTPLUG support. Disabling 'instance-first-boot' is not supported as we apply networking config too early in boot to have processed userdata (along with the fact that this would be a pretty big foot-gun). The concept of update events on datasource has been split into supported update events and default update events. Defaults will be used if there is no user-defined update events, but user-defined events won't be supplied if they aren't supported. When applying the networking config, we now check to see if the event is supported by the datasource as well as if it is enabled. Configuration looks like: updates: network: when: ['boot']
2021-05-11Update test characters in substitution unit test (#893)James Falcon
In newer versions of python, when using urllib.parse, lines containing newline or tab characters now get sanitized. This caused a unit test to fail. See https://bugs.python.org/issue43882
2021-05-07cc_disk_setup.py: remove UDEVADM_CMD definition as not used (#886)dermotbradley
UDEVADM_CMD is defined but not actually used in cc_disk_setup.py so remove it. Also modify the comment at top of read_parttbl function to remove the reference to udevadm which implies it is used to scan the partition table.
2021-05-07Add AlmaLinux OS support (#872)Andrew Lukoshko
AlmaLinux OS is RHEL-compatible so all the changes needed are trivial.
2021-05-05Release 21.2 (#890)James Falcon
Bump the version in cloudinit/version.py to 21.2 and update ChangeLog. LP: #1927254
2021-05-05Add \r\n check for SSH keys in Azure (#889)James Falcon
See https://bugs.launchpad.net/cloud-init/+bug/1910835
2021-05-03Revert "Add support to resize rootfs if using LVM (#721)" (#887)Daniel Watkins
This reverts commit 74fa008bfcd3263eb691cc0b3f7a055b17569f8b. During pre-release testing, we discovered two issues with this commit. Firstly, there's a typo in the udevadm command that causes a TypeError for _all_ growpart executions. Secondly, the LVM resizing does not appear to successfully resize everything up to the LV, though some things do get resized. We certainly want this change, so we'll be happy to review and land it alongside an integration test which confirms that it is working as expected. LP: #1922742
2021-04-27Azure: adding support for consuming userdata from IMDS (#884)Anh Vo
2021-04-26Fix chef module run failure when chef_license is set (#868)Ben Hughes
Move chef_license from TPL_PATH_KEYS to TPL_KEYS as the chef license setting is not a path but must be added to the client config template. Fixes file or folder not found exception raised from ensure_dirs.
2021-04-26Azure: Retry net metadata during nic attach for non-timeout errs (#878)aswinrajamannar
When network interfaces are hot-attached to the VM, attempting to get network metadata might return 410 (or 500, 503 etc) because the info is not yet available. In those cases, we retry getting the metadata before giving up. The only case where we can move on to wait for more nic attach events is if the call times out despite retries, which means the interface is not likely a primary interface, and we should try for more nic attach events.
2021-04-26Azure: Retrieve username and hostname from IMDS (#865)Thomas Stringer
This change allows us to retrieve the username and hostname from IMDS instead of having to rely on the mounted OVF.
2021-04-23Azure: eject the provisioning iso before reporting ready (#861)Anh Vo
Due to hyper-v implementations, iso ejection is more efficient if performed from within the guest. The code will attempt to perform a best-effort ejection. Failure during ejection will not prevent reporting ready from happening. If iso ejection is successful, later iso ejection from the platform will be a no-op. In the event the iso ejection from the guest fails, iso ejection will still happen at the platform level.
2021-04-22Use `partprobe` to re-read partition table if available (#856)Nicolas Bock
The blkdev command is fragile re-reading partition tables if a partition is mounted. This change instead uses the partprobe if it is available. LP: #1920939
2021-04-19fix error on upgrade caused by new vendordata2 attributes (#869)James Falcon
In #777, we added 'vendordata2' and 'vendordata2_raw' attributes to the DataSource class, but didn't use the upgrade framework to deal with an unpickle after upgrade. This commit adds the necessary upgrade code. Additionally, added a smaller-scope upgrade test to our integration tests that will be run on every CI run so we catch these issues immediately in the future. LP: #1922739
2021-04-15add prefer_fqdn_over_hostname config option (#859)hamalq
the above option allows the user to control the behavior of a distro hostname selection if both short hostname and FQDN are supplied. If `prefer_fqdn_over_hostname` is true the FQDN will be selected as hostname; if false the hostname will be selected LP: #1921004
2021-04-14doc: Replace remaining references to user-scripts as a config module (#866)Ryan Harper
git-grep showed a few more locations where we refer to a "user-scripts" config module which is really cc_scripts_user module. Replace these references with slightly different language so as not to confuse future me when looking for "user-scripts" vs. "scripts-user"
2021-04-13azure: Removing ability to invoke walinuxagent (#799)Anh Vo
Invoking walinuxagent from within cloud-init is no longer supported/necessary
2021-04-13Add Vultr support (#827)David Dymko
This PR adds in support so that cloud-init can run on instances deployed on Vultr cloud. This was originally brought up in #628. Co-authored-by: Eric Benner <ebenner@vultr.com>
2021-04-12Fix unpickle for source paths missing run_dir (#863)lucasmoura
On the datasource class, we require the use of paths.run_dir to perform some operations. On older cloud-init version, the Paths class does not have the run_dir attribute. To fix that, we are now manually adding that attribute in the Paths object if doesn't exist in the unpickle operation. LP: #1899299
2021-04-09sysconfig: use BONDING_MODULE_OPTS on SUSE (#831)Jens Sandmann
Update sysconfig configuration to use BONDING_MODULES_OPTS instead of BONDING_OPTS when on a SUSE system. The sysconfig support requires use of BONDING_MODULE_OPTS whereas the initscript support that rhel uses requires BONDING_OPTS.
2021-04-07bringup_static_routes: fix gateway check (#850)Petr Fedchenkov
When bringing up DHCP-provided static routes, we check for "0.0.0.0/0" to indicate an unspecified gateway. However, when parsing the static route in `parse_static_routes`, the gateway is never specified with a net length, so the "/0" will never happen. This change updates the gateway check to check only for "0.0.0.0".
2021-03-30Add support to resize rootfs if using LVM (#721)Eduardo Otubo
This patch adds support to resize a single partition of a VM if it's using an LVM underneath. The patch detects if it's LVM if the given block device is a device mapper by its name (e.g. `/dev/dm-1`) and if it has slave devices under it on sysfs. After that syspath is updated to the real block device and growpart will be called to resize it (and automatically its Physical Volume). The Volume Group will be updated automatically and a final call to extend the rootfs to the remaining space available will be made. Using the same growpart configuration, the user can specify only one device to be resized when using LVM and growpart, otherwise cloud-init won't know which one should be resized and will fail. rhbz: #1810878 LP: #1799953 Signed-off-by: Eduardo Otubo <otubo@redhat.com> Signed-off-by: Scott Moser <smoser@brickies.net>
2021-03-29Fix mis-detecting network configuration in initramfs cmdline (#844)James Falcon
klibc initramfs in debian allows the 'iscsi_target_ip=' cmdline parameter to specify an iscsi device attachment. This can cause cloud-init to mis-detect the cmdline paramter as a networking config. LP: #1919188
2021-03-25Azure helper: Ensure Azure http handler sleeps between retries (#842)Johnson Shi
Ensure that the Azure helper's http handler sleeps a fixed duration between retry failure attempts. The http handler will sleep a fixed duration between failed attempts regardless of whether the attempt failed due to (1) request timing out or (2) instant failure (no timeout). Due to certain platform issues, the http request to the Azure endpoint may instantly fail without reaching the http timeout duration. Without sleeping a fixed duration in between retry attempts, the http handler will loop through the max retry attempts quickly. This causes the communication between cloud-init and the Azure platform to be less resilient due to the short total duration if there is no sleep in between retries.
2021-03-19write passwords only to serial console, lock down cloud-init-output.log (#847)Daniel Watkins
Prior to this commit, when a user specified configuration which would generate random passwords for users, cloud-init would cause those passwords to be written to the serial console by emitting them on stderr. In the default configuration, any stdout or stderr emitted by cloud-init is also written to `/var/log/cloud-init-output.log`. This file is world-readable, meaning that those randomly-generated passwords were available to be read by any user with access to the system. This presents an obvious security issue. This commit responds to this issue in two ways: * We address the direct issue by moving from writing the passwords to sys.stderr to writing them directly to /dev/console (via util.multi_log); this means that the passwords will never end up in cloud-init-output.log * To avoid future issues like this, we also modify the logging code so that any files created in a log sink subprocess will only be owner/group readable and, if it exists, will be owned by the adm group. This results in `/var/log/cloud-init-output.log` no longer being world-readable, meaning that if there are other parts of the codebase that are emitting sensitive data intended for the serial console, that data is no longer available to all users of the system. LP: #1918303
2021-03-16Fix stack trace if vendordata_raw contained an array (#837)eb3095
The implementation in existing datasources means that vendordata_raw is not "raw" as it ideally would be. Instead, actual values may include bytes, string or list. If the value was a list, then the attempt to persist that data to a file in '_store_rawdata' would raise a TypeError. The change is to encode with util.json_dumps (which is safe for binary data) before writing.
2021-03-15archlinux: Fix broken locale logic (#841)Kristian Klausen
The locale wasn't persisted correct nor set. LP: #1402406
2021-03-08Fix requiring device-number on EC2 derivatives (#836)James Falcon
#342 (70dbccbb) introduced the ability to determine route-metrics based on the `device-number` provided by the EC2 IMDS. Not all datasources that subclass EC2 will have this attribute, so allow the old behavior if `device-number` is not present. LP: #1917875
2021-03-08net: exclude OVS internal interfaces in get_interfaces (#829)Daniel Watkins
`get_interfaces` is used to in two ways, broadly: firstly, to determine the available interfaces when converting cloud network configuration formats to cloud-init's network configuration formats; and, secondly, to ensure that any interfaces which are specified in network configuration are (a) available, and (b) named correctly. The first of these is unaffected by this commit, as no clouds support Open vSwitch configuration in their network configuration formats. For the second, we check that MAC addresses of physical devices are unique. In some OVS configurations, there are OVS-created devices which have duplicate MAC addresses, either with each other or with physical devices. As these interfaces are created by OVS, we can be confident that (a) they will be available when appropriate, and (b) that OVS will name them correctly. As such, this commit excludes any OVS-internal interfaces from the set of interfaces returned by `get_interfaces`. LP: #1912844
2021-03-03Add flexibility to IMDS api-version (#793)Thomas Stringer
Add flexibility to IMDS api-version by having both a desired IMDS api-version and a minimum api-version. The desired api-version will be used first, and if that fails it will fall back to the minimum api-version.