summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2020-10-15azure: clean up and refactor report_diagnostic_event (#563)Johnson Shi
This moves logging into `report_diagnostic_event`, to clean up its callsites.
2020-10-13net: add the ability to blacklist network interfaces based on driver during ↵Anh Vo
enumeration of physical network devices (#591)
2020-10-06integration_tests: don't error on cloud-init failure (#596)Daniel Watkins
pycloudlib's default behaviour is to raise an exception if cloud-init fails to run in an instance being launched. For cloud-init testing, we want our test assertions to flag up failures, so we disable this behaviour for instances we launch.
2020-10-06integration_tests: improve cloud-init.log assertions (#593)Daniel Watkins
2020-10-06conftest.py: remove top-level import of httpretty (#599)Daniel Watkins
This means that the integration tests do not need to install test-requirements.txt in order to successfully import `conftest.py`.
2020-10-05tox.ini: add integration-tests testenv definition (#595)Daniel Watkins
2020-10-05PULL_REQUEST_TEMPLATE.md: empty checkboxes need a space (#597)Daniel Watkins
2020-10-02add integration test for LP: #1886531 (#592)Daniel Watkins
2020-10-01Initial implementation of integration testing infrastructure (#581)James Falcon
2020-10-01Fix name of ntp and chrony service on CentOS and RHEL. (#589)Scott Moser
The service installed by the CentOS and RHEL 'ntp' package is ntpd.service not ntp.service Fix that for those two distros. Also fix chrony service from 'chrony' to 'chronyd'. LP: #1897915
2020-09-29Adding a PR template (#587)James Falcon
2020-09-24Azure parse_network_config uses fallback cfg when generate IMDS network cfg ↵Johnson Shi
fails (#549) Azure datasource's `parse_network_config` throws a fatal uncaught exception when an exception is raised during generation of network config from IMDS metadata. This happens when IMDS metadata is invalid/corrupted (such as when it is missing network or interface metadata). This causes the rest of provisioning to fail. This changes `parse_network_config` to be a non-fatal implementation. Additionally, when generating network config from IMDS metadata fails, fall back on generating fallback network config (`_generate_network_config_from_fallback_config`). This also changes fallback network config generation (`_generate_network_config_from_fallback_config`) to blacklist an additional driver: `mlx5_core`.
2020-09-23features: refresh docs for easier out-of-context reading (#582)Daniel Watkins
2020-09-22Fix typo in resolv_conf module's description (#578)Wacław Schiller
2020-09-22cc_users_groups: minor doc formatting fix (#577)Daniel Watkins
Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-09-21Fix typo in disk_setup module's description (#579)Wacław Schiller
2020-09-18Add vendor-data support to seedfrom parameter for NoCloud and OVF (#570)Johann Queuniet
2020-09-16boot.rst: add First Boot Determination section (#568)Daniel Watkins
LP: #1888858
2020-09-16opennebula.rst: minor readability improvements (#573)Mina Galić
- use `sh` as highlight language of the code block - change order so that the confusing indentation is less confusing
2020-09-16cloudinit: remove unused LOG variables (#574)Daniel Watkins
Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-09-15create a shutdown_command method in distro classes (#567)Emmanuel Thomé
Under FreeBSD, we want to use "shutdown -p" for poweroff. Alpine Linux also has some specificities. We choose to define a method that returns the shutdown command line to use, rather than a method that actually does the shutdown. This makes it easier to have the tests in test_handler_power_state do their verifications. Two tests are added for the special behaviours that are known so far.
2020-09-15user_data: remove unused constant (#566)Daniel Watkins
This was added in d00126c167fc06d913d99cfc184bf3402cb8cf53, but not removed in ef041fd822a2cf3a4022525e942ce988b1f95180 which removed the one usage of it from the original commit.
2020-09-15network: Fix type and respect name when rendering vlan in sysconfig. (#541)Eduardo Otubo
Prior to this change, vlans were rendered in sysconfig with 'TYPE=Ethernet', and incorrectly rendered the PHYSDEV based on the name of the vlan device rather than the 'link' provided in the network config. The change here fixes: * rendering of TYPE=Ethernet for a vlan * adds a warning if the configured device name is not supported per the RHEL 7 docs "11.5. Naming Scheme for VLAN Interfaces" LP: #1788915 LP: #1826608 RHBZ: #1861871
2020-09-10Retrieve SSH keys from IMDS first with OVF as a fallback (#509)Thomas Stringer
* pull ssh keys from imds first and fall back to ovf if unavailable * refactor log and diagnostic messages * refactor the OpenSSLManager instantiation and certificate usage * fix unit test where exception was being silenced for generate cert * fix tests now that certificate is not always generated * add documentation for ssh key retrieval * add ability to check if http client has security enabled * refactor certificate logic to GoalState
2020-09-08Add jqueuniet as contributor (#569)Johann Queuniet
2020-09-08distros: minor typo fix (#562)Daniel Watkins
Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-09-03Bump the integration-requirements versioned dependencies (#565)Paride Legovini
During the ec2 integration test runs we occasionally see failures in deleting test instances. Hopefully a newer boto3 will be more robust. Also bump: paramiko, cryptography, pylxd (now pulling it from pypi). Tested with a full Xenial EC2 cloud_tests run.
2020-09-01network-config-format-v1: fix typo in nameserver example (#564)Stanislas
2020-08-31Run cloud-init-local.service after the hv_kvp_daemon (#505)Robert Schweikert
This ensures that KVP data can be correctly reported to Azure.
2020-08-28Add method type hints for Azure helper (#540)Johnson Shi
This reverts commit 8d25d5e6fac39ab3319ec5d37d23196429fb0c95.
2020-08-27systemd: add Before=shutdown.target when Conflicts=shutdown.target is used ↵Paride Legovini
(#546) Lintian spotted the following issue: The specified systemd .service file contains both DefaultDependencies=no and Conflicts=shutdown.target directives without Before=shutdown.target. This can lead to problems during shutdown because the service may linger until the very end of shutdown sequence as nothing requests to stop it before (due to DefaultDependencies=no). There is race condition between stopping units and systemd getting a request to exit the main loop, so it may proceed with shutdown before all pending stop jobs have been processed. Please add Before=shutdown.target. [1] https://lintian.debian.org/tags/systemd-service-file-shutdown-problems.html Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-08-27LXD: detach network from profile before deleting it (#542)Paride Legovini
* LXD: detach network from profile before deleting it When cleaning up the bridge network created by default by LXD as part of the `lxd init` process detach the network its profile before deleting it. LXD will otherwise refuse to delete it with error: Error: The network is currently in use. Discussion with LXD upstream: https://github.com/lxc/lxd/issues/7804. LP: #1776958 * LXD bridge deletion: fail if bridge exists but can't be deleted * LXD bridge deletion: remove useless failure logging
2020-08-27redhat spec: add missing BuildRequires (#552)Paride Legovini
456fb55744a1acc6bd2f464b7656a9c33d0b7ac5 made tools/read-dependencies and package/brpm distinguish between build dependencies and runtime dependencies, however packages/redhat/cloud-init.spec.in expects all the dependencies to be in the 'requires' list, thus missing some build dependencies. This change makes cloud-init.spec use 'buildrequires' too. The build happens to succeed without python3-devel on the epel-8 copr chroot as it pulls in the epel-rpm-macros package, which in turn depends on python3-devel. In other words the dependency is satisfied by chance. Packages building for Python 3 need to explicitly specify BuildRequires: python3-devel, see: [1]. [1] https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/
2020-08-26util: remove debug statement (#556)Joshua Powers
2020-08-26Fix cloud config on chef example (#551)lucasmoura
2020-08-25Release 20.3 (#547)20.3James Falcon
Bump the version in cloudinit/version.py to 20.3 and update ChangeLog. LP: #1892878
2020-08-25tox: bump the pylint version to 2.6.0 in the default run (#544)Paride Legovini
Changes: tox: bump the pylint version to 2.6.0 in the default run Fix pylint 2.6.0 W0707 warnings (raise-missing-from)
2020-08-24Azure: Add netplan driver filter when using hv_netvsc driver (#539)James Falcon
This fixes a long delay during boot of some instances. For Azure instance types using SR-IOV via the Hyper-V netvsc network driver, two network interfaces are created that share the same MAC, but only the virtual device should be configured and used. Updating the netplan configuration to filter on the hv_netvsc driver prevents netplan from trying to figure both devices. LP: #1830740
2020-08-24query: do not handle non-decodable non-gzipped content (#543)Chad Smith
2020-08-24DHCP sandboxing failing on noexec mounted /var/tmp (#521)Eduardo Otubo
* DHCP sandboxing failing on noexec mounted /var/tmp If /var/tmp is mounted with noexec option the DHCP sandboxing will fail with Permission Denied. This patch simply avoids this error by checking the exec permission updating the dhcp path in negative case. rhbz: https://bugzilla.redhat.com/show_bug.cgi?id=1857309 Signed-off-by: Eduardo Otubo <otubo@redhat.com> * Replacing with os.* calls * Adding test and removing isfile() useless call. Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-08-21Update the list of valid ssh keys. (#487)Ole-Martin Bratteng
Update ssh_util.py with latest list of keys (from openssh-8.3p1/sshkey.c), Added keys: sk-ecdsa-sha2-nistp256-cert-v01@openssh.com sk-ecdsa-sha2-nistp256@openssh.com sk-ssh-ed25519-cert-v01@openssh.com sk-ssh-ed25519@openssh.com ssh-xmss-cert-v01@openssh.com ssh-xmss@openssh.com LP: #1877869
2020-08-20cmd: cloud-init query to handle compressed userdata (#516)Chad Smith
cloud-init query tries to directly load and decode raw user-data from /var/lib/cloud/instance/user-data.txt. This results in UnicodeDecodeErrors on some platforms which provide compressed content. Avoid UnicodeDecoderErrors when parsing compressed user-data at /var/lib/cloud/instance/user-data.txt. LP: #1889938
2020-08-20Pushing cloud-init log to the KVP (#529)Moustafa Moustafa
Push the cloud-init.log file (Up to 500KB at once) to the KVP before reporting ready to the Azure platform. Based on the analysis done on a large sample of cloud-init.log files, Here's the statistics collected on the log file size: P50 P90 P95 P99 P99.9 P99.99 137K 423K 537K 3.5MB 6MB 16MB This change limits the size of cloud-init.log file data that gets dumped to KVP to 500KB. So for ~95% of the cases, the whole log file will be dumped and for the remaining ~5%, we will get the last 500KB of the cloud-init.log file. To asses the performance of the 500KB limit, 250 VM were deployed with a 500KB cloud-init.log file and the time taken to compress, encode and dump the entries to KVP was measured. Here's the time in milliseconds percentiles: P50 P99 P999 75.705 232.701 1169.636 Another 250 VMs were deployed with this logic dumping their normal cloud-init.log file to KVP, the same timing was measured as above. Here's the time in milliseconds percentiles: P50 P99 P999 1.88 5.277 6.992 Added excluded_handlers to the report_event function to be able to opt-out from reporting the events of the compressed cloud-init.log file to the cloud-init.log file. The KVP break_down logic had a bug, where it will reuse the same key for all the split chunks of KVP which results in overwriting the split KVPs by the last one when consumed by Hyper-V. I added the split chunk index as a differentiator to the KVP key. The Hyper-V consumes the KVPs from the KVP file as chunks whose key is 512KB and value is 2048KB but the Azure platform expects the value to be 1024KB, thus I introduced the Azure value limit.
2020-08-19Add Alpine Linux support. (#535)dermotbradley
Add new module cc_apk_configure for creating Alpine /etc/apk/repositories file. Modify cc_ca_certs, cc_ntp, cc_power_state_change, and cc_resolv_conf for Alpine. Add Alpine template files for Chrony and Busybox NTP support. Add Alpine template file for /etc/hosts.
2020-08-18Detect kernel version before swap file creation (#428)Eduardo Otubo
According to man page `man 8 swapon', "Preallocated swap files are supported on XFS since Linux 4.18". This patch checks for kernel version before attepting to create swapfile, using dd for XFS only on kernel versions <= 4.18 or btrfs. Add new func util.kernel_version which returns a tuple of ints (major, minor) Signed-off-by: Eduardo Otubo otubo@redhat.com
2020-08-17cli: add devel make-mime subcommand (#518)Ryan Harper
* cli: add devel make-mime subcommand Cloud-init documents an in-source-tree tool, make-mime.py used to help users create multi-part mime user-data. This tool is not shipped in the cloud-init install and unavailable at runtime. This patch takes tools/make-mime.py and makes the functionality available via the devel subcommand. The primary interface of --attach file:content-type is still present. The cli now adds: -l, --list-types Print out a list of supported content-types -f, --force Ignore errors for unsupported content-types The tool will now raise a RunTime error if the supplied content-type is not supported (or more likely a typo: x-shell-script vs. x-shellscript) * make-mime: write to stderr and exit 1 instead of raising RuntimeError * Update example to match docs * Update docs for make-mime subcommand * Remove tools/make-mime.py; replaced by cloud-init devel make-mime Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-08-14user-data: only verify mime-types for TYPE_NEEDED and x-shellscript (#511)Ryan Harper
Commit d00126c167fc06d913d99cfc184bf3402cb8cf53 regressed cloud-init handling in multipart MIME user-data. Specifically, cloud-init would examine the payload of the MIME part to determine what the content type and subsequently which handler to use. This meant that user-data which had shellscript payloads (starts with #!) were always handled as shellscripts, rather than their declared MIME type and affected when the payload was handled. One failing scenario was a MIME part with text/cloud-boothook type declared and a shellscript payload. This was run at shellscript processing time rather than boothook time resulting in an change in behavior from previous cloud-init releases. To continue to support known scenarios where clouds have specifed a MIME type of text/x-shellscript but provided a payload of something other than shellscripts, we're changing the lookup logic to check for the TYPES_NEEDED (text/plain, text/x-not-multipart) and only text/x-shellscript. It is safe to check text/x-shellscript parts as all shellscripts must include the #! marker and will be detected as text/x-shellscript types. If the content is missing the #! marker, it will not be excuted. If the content is detected as something cloud-init supports, such as #cloud-config the appropriate cloud-init handler will be used. This change will fix hanldling for parts which were shellscripts but ran with the wrong handler due to ignoring of the provided mime-type. LP: #1888822
2020-08-14DataSourceOracle: retry twice (and document why we retry at all) (#536)Daniel Watkins
2020-08-13Refactor Azure report ready code (#468)Johnson Shi
This PR refactors Azure report ready code to include more robust tests and telemetry.
2020-08-13tox.ini: pin correct version of httpretty in xenial{,-dev} envs (#531)Daniel Watkins
The version was bumped in c7248059dd2faaaadfbcef5c83e8e8ea166d6767 to support running on Python 3.7+ systems. Now that we have separate `xenial` and `xenial-dev` tox environments, we can restore the correct pinning for `xenial` without breaking `xenial-dev` on developer machines. Also drop the `mock` dependency from `xenial-shared-deps`; its removal was missed in 5f8f85bb38cc972d3d2c705a1ec73db3f690f323.
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-10 23:37:49 +0000