summaryrefslogtreecommitdiff
path: root/cloudinit
AgeCommit message (Collapse)Author
2020-09-24Azure parse_network_config uses fallback cfg when generate IMDS network cfg ↵Johnson Shi
fails (#549) Azure datasource's `parse_network_config` throws a fatal uncaught exception when an exception is raised during generation of network config from IMDS metadata. This happens when IMDS metadata is invalid/corrupted (such as when it is missing network or interface metadata). This causes the rest of provisioning to fail. This changes `parse_network_config` to be a non-fatal implementation. Additionally, when generating network config from IMDS metadata fails, fall back on generating fallback network config (`_generate_network_config_from_fallback_config`). This also changes fallback network config generation (`_generate_network_config_from_fallback_config`) to blacklist an additional driver: `mlx5_core`.
2020-09-23features: refresh docs for easier out-of-context reading (#582)Daniel Watkins
2020-09-22Fix typo in resolv_conf module's description (#578)Wacław Schiller
2020-09-22cc_users_groups: minor doc formatting fix (#577)Daniel Watkins
Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-09-21Fix typo in disk_setup module's description (#579)Wacław Schiller
2020-09-18Add vendor-data support to seedfrom parameter for NoCloud and OVF (#570)Johann Queuniet
2020-09-16cloudinit: remove unused LOG variables (#574)Daniel Watkins
Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-09-15create a shutdown_command method in distro classes (#567)Emmanuel Thomé
Under FreeBSD, we want to use "shutdown -p" for poweroff. Alpine Linux also has some specificities. We choose to define a method that returns the shutdown command line to use, rather than a method that actually does the shutdown. This makes it easier to have the tests in test_handler_power_state do their verifications. Two tests are added for the special behaviours that are known so far.
2020-09-15user_data: remove unused constant (#566)Daniel Watkins
This was added in d00126c167fc06d913d99cfc184bf3402cb8cf53, but not removed in ef041fd822a2cf3a4022525e942ce988b1f95180 which removed the one usage of it from the original commit.
2020-09-15network: Fix type and respect name when rendering vlan in sysconfig. (#541)Eduardo Otubo
Prior to this change, vlans were rendered in sysconfig with 'TYPE=Ethernet', and incorrectly rendered the PHYSDEV based on the name of the vlan device rather than the 'link' provided in the network config. The change here fixes: * rendering of TYPE=Ethernet for a vlan * adds a warning if the configured device name is not supported per the RHEL 7 docs "11.5. Naming Scheme for VLAN Interfaces" LP: #1788915 LP: #1826608 RHBZ: #1861871
2020-09-10Retrieve SSH keys from IMDS first with OVF as a fallback (#509)Thomas Stringer
* pull ssh keys from imds first and fall back to ovf if unavailable * refactor log and diagnostic messages * refactor the OpenSSLManager instantiation and certificate usage * fix unit test where exception was being silenced for generate cert * fix tests now that certificate is not always generated * add documentation for ssh key retrieval * add ability to check if http client has security enabled * refactor certificate logic to GoalState
2020-09-08distros: minor typo fix (#562)Daniel Watkins
Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-08-28Add method type hints for Azure helper (#540)Johnson Shi
This reverts commit 8d25d5e6fac39ab3319ec5d37d23196429fb0c95.
2020-08-27LXD: detach network from profile before deleting it (#542)Paride Legovini
* LXD: detach network from profile before deleting it When cleaning up the bridge network created by default by LXD as part of the `lxd init` process detach the network its profile before deleting it. LXD will otherwise refuse to delete it with error: Error: The network is currently in use. Discussion with LXD upstream: https://github.com/lxc/lxd/issues/7804. LP: #1776958 * LXD bridge deletion: fail if bridge exists but can't be deleted * LXD bridge deletion: remove useless failure logging
2020-08-26util: remove debug statement (#556)Joshua Powers
2020-08-25Release 20.3 (#547)20.3James Falcon
Bump the version in cloudinit/version.py to 20.3 and update ChangeLog. LP: #1892878
2020-08-25tox: bump the pylint version to 2.6.0 in the default run (#544)Paride Legovini
Changes: tox: bump the pylint version to 2.6.0 in the default run Fix pylint 2.6.0 W0707 warnings (raise-missing-from)
2020-08-24Azure: Add netplan driver filter when using hv_netvsc driver (#539)James Falcon
This fixes a long delay during boot of some instances. For Azure instance types using SR-IOV via the Hyper-V netvsc network driver, two network interfaces are created that share the same MAC, but only the virtual device should be configured and used. Updating the netplan configuration to filter on the hv_netvsc driver prevents netplan from trying to figure both devices. LP: #1830740
2020-08-24query: do not handle non-decodable non-gzipped content (#543)Chad Smith
2020-08-24DHCP sandboxing failing on noexec mounted /var/tmp (#521)Eduardo Otubo
* DHCP sandboxing failing on noexec mounted /var/tmp If /var/tmp is mounted with noexec option the DHCP sandboxing will fail with Permission Denied. This patch simply avoids this error by checking the exec permission updating the dhcp path in negative case. rhbz: https://bugzilla.redhat.com/show_bug.cgi?id=1857309 Signed-off-by: Eduardo Otubo <otubo@redhat.com> * Replacing with os.* calls * Adding test and removing isfile() useless call. Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-08-21Update the list of valid ssh keys. (#487)Ole-Martin Bratteng
Update ssh_util.py with latest list of keys (from openssh-8.3p1/sshkey.c), Added keys: sk-ecdsa-sha2-nistp256-cert-v01@openssh.com sk-ecdsa-sha2-nistp256@openssh.com sk-ssh-ed25519-cert-v01@openssh.com sk-ssh-ed25519@openssh.com ssh-xmss-cert-v01@openssh.com ssh-xmss@openssh.com LP: #1877869
2020-08-20cmd: cloud-init query to handle compressed userdata (#516)Chad Smith
cloud-init query tries to directly load and decode raw user-data from /var/lib/cloud/instance/user-data.txt. This results in UnicodeDecodeErrors on some platforms which provide compressed content. Avoid UnicodeDecoderErrors when parsing compressed user-data at /var/lib/cloud/instance/user-data.txt. LP: #1889938
2020-08-20Pushing cloud-init log to the KVP (#529)Moustafa Moustafa
Push the cloud-init.log file (Up to 500KB at once) to the KVP before reporting ready to the Azure platform. Based on the analysis done on a large sample of cloud-init.log files, Here's the statistics collected on the log file size: P50 P90 P95 P99 P99.9 P99.99 137K 423K 537K 3.5MB 6MB 16MB This change limits the size of cloud-init.log file data that gets dumped to KVP to 500KB. So for ~95% of the cases, the whole log file will be dumped and for the remaining ~5%, we will get the last 500KB of the cloud-init.log file. To asses the performance of the 500KB limit, 250 VM were deployed with a 500KB cloud-init.log file and the time taken to compress, encode and dump the entries to KVP was measured. Here's the time in milliseconds percentiles: P50 P99 P999 75.705 232.701 1169.636 Another 250 VMs were deployed with this logic dumping their normal cloud-init.log file to KVP, the same timing was measured as above. Here's the time in milliseconds percentiles: P50 P99 P999 1.88 5.277 6.992 Added excluded_handlers to the report_event function to be able to opt-out from reporting the events of the compressed cloud-init.log file to the cloud-init.log file. The KVP break_down logic had a bug, where it will reuse the same key for all the split chunks of KVP which results in overwriting the split KVPs by the last one when consumed by Hyper-V. I added the split chunk index as a differentiator to the KVP key. The Hyper-V consumes the KVPs from the KVP file as chunks whose key is 512KB and value is 2048KB but the Azure platform expects the value to be 1024KB, thus I introduced the Azure value limit.
2020-08-19Add Alpine Linux support. (#535)dermotbradley
Add new module cc_apk_configure for creating Alpine /etc/apk/repositories file. Modify cc_ca_certs, cc_ntp, cc_power_state_change, and cc_resolv_conf for Alpine. Add Alpine template files for Chrony and Busybox NTP support. Add Alpine template file for /etc/hosts.
2020-08-18Detect kernel version before swap file creation (#428)Eduardo Otubo
According to man page `man 8 swapon', "Preallocated swap files are supported on XFS since Linux 4.18". This patch checks for kernel version before attepting to create swapfile, using dd for XFS only on kernel versions <= 4.18 or btrfs. Add new func util.kernel_version which returns a tuple of ints (major, minor) Signed-off-by: Eduardo Otubo otubo@redhat.com
2020-08-17cli: add devel make-mime subcommand (#518)Ryan Harper
* cli: add devel make-mime subcommand Cloud-init documents an in-source-tree tool, make-mime.py used to help users create multi-part mime user-data. This tool is not shipped in the cloud-init install and unavailable at runtime. This patch takes tools/make-mime.py and makes the functionality available via the devel subcommand. The primary interface of --attach file:content-type is still present. The cli now adds: -l, --list-types Print out a list of supported content-types -f, --force Ignore errors for unsupported content-types The tool will now raise a RunTime error if the supplied content-type is not supported (or more likely a typo: x-shell-script vs. x-shellscript) * make-mime: write to stderr and exit 1 instead of raising RuntimeError * Update example to match docs * Update docs for make-mime subcommand * Remove tools/make-mime.py; replaced by cloud-init devel make-mime Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-08-14user-data: only verify mime-types for TYPE_NEEDED and x-shellscript (#511)Ryan Harper
Commit d00126c167fc06d913d99cfc184bf3402cb8cf53 regressed cloud-init handling in multipart MIME user-data. Specifically, cloud-init would examine the payload of the MIME part to determine what the content type and subsequently which handler to use. This meant that user-data which had shellscript payloads (starts with #!) were always handled as shellscripts, rather than their declared MIME type and affected when the payload was handled. One failing scenario was a MIME part with text/cloud-boothook type declared and a shellscript payload. This was run at shellscript processing time rather than boothook time resulting in an change in behavior from previous cloud-init releases. To continue to support known scenarios where clouds have specifed a MIME type of text/x-shellscript but provided a payload of something other than shellscripts, we're changing the lookup logic to check for the TYPES_NEEDED (text/plain, text/x-not-multipart) and only text/x-shellscript. It is safe to check text/x-shellscript parts as all shellscripts must include the #! marker and will be detected as text/x-shellscript types. If the content is missing the #! marker, it will not be excuted. If the content is detected as something cloud-init supports, such as #cloud-config the appropriate cloud-init handler will be used. This change will fix hanldling for parts which were shellscripts but ran with the wrong handler due to ignoring of the provided mime-type. LP: #1888822
2020-08-14DataSourceOracle: retry twice (and document why we retry at all) (#536)Daniel Watkins
2020-08-13Refactor Azure report ready code (#468)Johnson Shi
This PR refactors Azure report ready code to include more robust tests and telemetry.
2020-08-13Support Oracle IMDSv2 API (#528)James Falcon
* v2 of the API is now default with fallback to v1. * Refactored the Oracle datasource to fetch version, instance, and vnic metadata simultaneously.
2020-08-13Fix 'Users and Groups' configuration documentation (#530)sshedi
Few of the 'User and Groups' configurations in cloud-config have no effect on already existing users. This was not documented earlier. This change set adds that information to documentation. Signed-off-by: Shreenidhi Shedi <sshedi@vmware.com>
2020-08-13cloudinit.distros: update docstrings of add_user and create_user (#527)Daniel Watkins
This aligns their docstrings more closely with their actual behaviour.
2020-08-10DataSourceOracle: refactor to use only OPC v1 endpoint (#493)Daniel Watkins
The /opc/v1/ metadata endpoints[0] are universally available in Oracle Cloud Infrastructure and the OpenStack endpoints are considered deprecated, so we can refactor the data source to use the OPC endpoints exclusively. This simplifies the datasource code substantially, and enables use of OPC-specific attributes in future. [0] https://docs.cloud.oracle.com/en-us/iaas/Content/Compute/Tasks/gettingmetadata.htm
2020-08-10Fix a typo in apt pipelining module (#525)Xiao Liang
* Fix a typo in apt pipelining module Changed `whcih` to `which`. * Update .github-cla-signers I have signed the CLA on Canonical's site, adding my username to list of CLA signers. * Update .github-cla-signers I need to sort the list alphabetically.
2020-08-07Recognize LABEL_FATBOOT labels (#513)James Falcon
Update DataSourceNoCloud and ds-identify to recognize LABEL_FATBOOT labels from blkid. Also updated associated tests. LP: #1841466
2020-08-06Handle additional identifier for SLES For HPC (#520)Robert Schweikert
Add "sle_hpc" to list of values which are variant 'suse'.
2020-07-22azure: disable bouncing hostname when setting hostname fails (#494)Anh Vo
DataSourceAzure: Gracefully handle the case of set hostname failure during provisioning
2020-07-21VMware: Support parsing DEFAULT-RUN-POST-CUST-SCRIPT (#441)xiaofengw-vmware
Add support for VMware's vCD configuration setting DEFAULT-RUN-POST-CUST-SCRIPT. When set True, it will default vms to run post customization scripts if the VM has not been configured in VMTools with "enable-custom-scripts" set False. Add datasource documentation with a bit more context about this interaction on VMware products. With this fix, the behavior will be: * If VM administrator doesn't want others to execute a script on this VM, VMtools can set "enable-custom-scripts" to false from the utility "vmware-toolbox-cmd". * If VM administrator doesn't set value to "enable-custom-scripts", then by default this script is disabled for security purpose. * For VMware's vCD product , the preference is to enable the script if "enable-custom-scripts" is not set. vCD will generate a configuration file with "DEFAULT-RUN-POST-CUST-SCRIPT" set to true. This flag works for both VMware customization engine and cloud-init.
2020-07-15DataSourceAzure: Use ValueError when JSONDecodeError is not available (#490)Anh Vo
JSONDecodeError is only available in Python 3.5+. When it isn't available (i.e. on Python 3.4, which cloud-init still supports) use the more generic ValueError.
2020-07-15cc_ca_certs.py: fix blank line problem when removing CAs and adding new one ↵dermotbradley
(#483) Problem: When cc_ca_certs configuration has both "remove-defaults: true" and also specifies one, or more, new trusted CAs to add then the resultant /etc/ca-certificates.conf file's 1st line is blank. As noted in comments in the existing cc_ca_certs.py code blank lines in this file cause problems. Fix: Before adding the cloud-init CA filename to this file first check the size of the file - if is is empty (as all existing CAs have been deleted) then write only the cloud-init CA filename to the file rather than appending it to the file.
2020-07-15ssh exit with non-zero status on disabled user (#472)Eduardo Otubo
It is confusing for scripts, where a disabled user has been specified, that ssh exits with a zero status by default without indication anything failed. I think exitting with a non-zero status would make more clear in scripts and automated setups where things failed, thus making noticing the issue and debugging easier. Signed-off-by: Eduardo Otubo <otubo@redhat.com> Signed-off-by: Aleksandar Kostadinov <akostadi@redhat.com> LP: #1170059
2020-07-15cloudinit: remove global disable of pylint W0107 and fix errors (#489)Daniel Watkins
* cloudinit: remove global disable of pylint W0107 and fix errors This includes removing a test class which contained no tests but wasn't detected as empty because of an errant pass statement. * .pylintrc: update disable comment to match arguments
2020-07-14networking: refactor wait_for_physdevs from cloudinit.net (#466)Daniel Watkins
* Refactor `cloudinit.net.wait_for_physdevs` to `cloudinit.distros.networking.Networking.wait_for_physdevs` * Split the Linux-specific `udevadm_settle` call out to a separate abstract `Networking.settle` method; implement it on `LinuxNetworking` and add a `NotImplementedError` implementation to `BSDNetworking` * Modify `wait_for_physdevs`s one callsite to use the new location LP: #1884626
2020-07-13cloudinit: remove global disable of pylint W0105 and fix errors (#480)Daniel Watkins
This includes a fix to a test that had a string concatenation issue, and so was only testing a prefix of what was intended.
2020-07-13Fix two minor warnings (#475)Daniel Watkins
2020-07-10cc_mounts: handle missing fstab (#484)Ryan Harper
Do not fail if /etc/fstab is not present. Some images, like container rootfs may not include this file by default. LP: #1886531
2020-07-06cloudinit: fix tip-pylint failures and bump pinned pylint version (#478)Daniel Watkins
Specifically: * disable E1102 in cloudinit/sources/helpers/openstack.py for reasons described in a comment, and * refactor `abs_join` to require at least one positional argument; this matches os.path.join's signature, and that mismatch is what was causing pylint to emit a warning * bump to pylint 2.4.2
2020-07-02tests: use markers to configure disable_subp_usage (#473)Daniel Watkins
This is an improvement over indirect parameterisation for a few reasons: * The test code is much easier to read, the mark names are much more intuitive than the indirect parameterisation invocation, and there's less boilerplate to boot * The fixture no longer has to overload the single parameter that fixtures can take with multiple meanings
2020-06-30Disable ec2 mirror for non aws instances (#390)lucasmoura
For versions before 20.2, we allowed the use of ec2 mirrors if the datasource availability_zone matches one of the ec2 regions. We are now updating that behavior to allow allow the use of ec2 mirrors on ec2 instances or if the user directly passes an an ec2 mirror url through #cloud-config apt directives. LP: #1456277
2020-06-30networking: refactor is_physical from cloudinit.net (#457)Daniel Watkins
As the first refactor PR, this also includes the initial structure for tests. LP: #1884619
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-28 00:16:58 +0000