summaryrefslogtreecommitdiff
path: root/tests
AgeCommit message (Collapse)Author
2020-01-16util: rename get_architecture to get_dpkg_architecture (#173)Daniel Watkins
This makes it clearer that we should only use this in code paths that will definitely have dpkg available to them. - Rename get_architecture -> get_dpkg_architecture - Add docstring to get_dpkg_architecture
2020-01-16Ensure util.get_architecture() runs only once (#172)Ryan Harper
* Ensure util.get_architecture() runs only once util.get_architecture() recently was wrapped using python3's lru_cache() which will cache the result so we only invoke 'dpkg --print-architecture' once. In practice, cloud-init.log will show multiple invocations of the command. The source of this was that the debian Distro object implements the get_primary_arch() with this command, but it was not calling it from util, but issuing a util.subp() directly. This branch also updates cc_apt_configure methods to fetch the arch value from the distro class, and then ensure that the methods apt_configure calls pass the arch value around. * utils: remove lsb_release and get_architecture wrappers The original lsb_release wrapper was used to prevent polluting the single value we cached, however lru_cache() already handles this case by using args, kwargs values to cache different calls to the method. * rename_apt_list: use all positional parameters
2020-01-14Only use gpart if it is the BSD gpart (#131)Conrad Hoffmann
Currently, cloud-init will happily try to run `gpart` on Linux even though on most distributions this a different tool [1]. Extend the availability check to make sure the `gpart` present is really the BSD variant, to avoid accidental execution. Also add a pointer to the docs, so that people do not try to install gpart on Linux in the expectation it will work with this module. [1] https://github.com/baruch/gpart
2020-01-10ssh_auth_key_fingerprints_disable test: fix capitalization (#165)Paride Legovini
Adapt the test to the new capitalization introduced in 8116493950e7c47af0ce66fc1bb5d799ce5e477a.
2020-01-08net: fix rendering of 'static6' in network config (#77)Ryan Harper
* net: fix rendering of 'static6' in network config A V1 static6 network typo was misrendered in eni, it's not valid. It was ignored in sysconfig and netplan. This branch fixes eni, updates sysconfig, netplan to render it correctly and adds unittests for all cases. Reported-by: Raphaël Enrici LP: #1850988 * net: add comment about static6 type in subnet_is_ipv6 Co-authored-by: Chad Smith <blackboxsw@gmail.com> Co-authored-by: Daniel Watkins <daniel@daniel-watkins.co.uk>
2019-12-20modules: drop cc_snap_config config module (#134)Chad Smith
cloud-init has moved to cc_snap module and a top-level config key 'snap'. cc_snap_config was deprecated in cloud-init version 18.2 Co-authored-by: Daniel Watkins <daniel@daniel-watkins.co.uk>
2019-12-20freebsd: introduce the freebsd renderer (#61)Gonéri Le Bouder
* freebsd: introduce the freebsd renderer Refactoring of the FreeBSD code base to provide a real network renderer for FreeBSD. Use the generic update_sysconfig_file() from rhel_util to handle the access to /etc/rc.conf. Interfaces are not automatically renamed by FreeBSD using the following configuration in /etc/rc.conf: ``` ifconfig_fxp0_name="eth0" ``` * freesd: use regex named groups Reduce the complexity of `get_interfaces_by_mac_on_freebsd()` with named groups. * freebsd: breaks up _write_network() in tree small functions - `_write_ifconfig_entries()` - `_write_route_entries()` - `_write_resolve_conf()` * extend find_fallback_nic() to support FreeBSD this uses `route -n show default` to find the default interface * freebsd: use dns keys from NetworkState class The NetworkState class (settings instance) exposes the DNS configuration in two keys: - `dns_nameservers` - `dns_searchdomains` On OpenStack, these keys are set when a global DNS server is set. The alternative is the `dns_nameservers` and `dns_search` keys from each subdomain. We continue to read those. * freebsd: properly target the /etc/resolv.conf file * freebsd: ignore 'service routing restart' ret code On FreeBSD 10, the restart of routing and dhclient is likely to fail because - routing: it cannot remove the loopback route, but it will still set up the default route as expected. - dhclient: it cannot stop the dhclient started by the netif service. In both case, the situation is ok, and we can proceed. * freebsd: handle case when metadata MAC local locally Handle the case where the metadata configuration comes with a MAC that does not exist locally. See: - https://github.com/canonical/cloud-init/pull/61/files/635ce14b3153934ba1041be48b7245062f21e960#r359600604 - https://github.com/canonical/cloud-init/pull/61/files/635ce14b3153934ba1041be48b7245062f21e960#r359600966 * freebsd: show up a warning if several subnet found The FreeBSD provider currently only allow one subnet per interface. * freebsd: honor the target parameter in _write_network * freebsd: log when a bad route is found * freebsd: pass _postcmds to start_services() * freebsd: updatercconf() is depercated Replace `updatercconf()` by `rhel_util.update_sysconfig_file()`. * freebsd: ensure gateway is ipv4 before using it With the legacy ENI format, an IPv6 gateway may be pushed. This instead of the expected IPv4. * freebsd: find_fallback_nic, support FB10 On FreeBSD <= 10, `ifconfig -l` ignores the down interfaces. * freebsd: use util.target_path() to load resolv.conf Ensure we access `/etc/resolv.conf`, not `etc/resolv.conf`. * freebsd: skip subnet without netmask Those are likely to be either invalid of in IPv6 format. IPv6 support will be addressed later in a new patchset. * freebsd: get_devicelist returns netif list Ensure `get_devicelist()` returns the list of known netif on FreeBSD. * replace rhel_util.update_sysconfig_file wrapper call, with a wrapper function * reverse if condition to remove an indent Co-authored-by: Igor Galić <me+github@igalic.co>
2019-12-19cc_snappy: remove deprecated module (#127)Daniel Watkins
* cc_snappy: remove deprecated module * cloud_tests: remove cc_snappy tests (and references) This module was deprecated in favor of cc_snap in cloud-init v.18.2
2019-12-18cloud-init: fix capitalisation of SSH (#126)Daniel Watkins
* cc_ssh: fix capitalisation of SSH * doc: fix capitalisation of SSH * cc_keys_to_console: fix capitalisation of SSH * ssh_util: fix capitalisation of SSH * DataSourceIBMCloud: fix capitalisation of SSH * DataSourceAzure: fix capitalisation of SSH * cs_utils: fix capitalisation of SSH * distros/__init__: fix capitalisation of SSH * cc_set_passwords: fix capitalisation of SSH * cc_ssh_import_id: fix capitalisation of SSH * cc_users_groups: fix capitalisation of SSH * cc_ssh_authkey_fingerprints: fix capitalisation of SSH
2019-12-18tests: Add tests for value of dsname in datasourcesAdam Dobrawy
2019-12-12azure: avoid re-running cloud-init when instance-id is byte-swapped (#84)AOhassan
Azure stores the instance ID with an incorrect byte ordering for the first three hyphen delimited parts. This results in invalid is_new_instance checks forcing Azure datasource to recrawl the metadata service. When persisting instance-id from the metadata service, swap the instance-id string byte order such that it is consistent with that returned by dmi information. Check whether the instance-id string is a byte-swapped match when determining correctly whether the Azure platform instance-id has actually changed.
2019-12-11ds_identify: if /sys is not available use dmidecode (#42)Igor Galić
On non-Linux systems, `/sys` won't be available. In these cases, we can query `dmidecode(8)` directly. This PR implements a dmi_decode function to query the same fields ds-identify would otherwise read from /sys. This path is taken when /sys isn't present. In addition to adding dmidecode support, non-Linux systems also need to map in virtualization detection as systemd-detect-virt is not present; on FreeBSD, use sysctl kern.vm_guest and provide a mapping[1] between BSD values and those that match with systemd-detect-virt[2]. 1. https://github.com/freebsd/freebsd/blob/master/sys/kern/subr_param.c#L149-L157 2. https://www.freedesktop.org/software/systemd/man/systemd-detect-virt.html LP: #1852442
2019-12-05Multiple file fix for AuthorizedKeysFile config (#60)Eduardo Otubo
Currently cloud-init does not know how to handle multiple file configuration on section AuthorizedKeysFile of ssh configuration. cloud-init will mess up the home user directory by creating bogus folders inside it. This patch provides a fix for this erroneous behavior. It gathers all keys from all the files listed on the section AuthorizedKeysFile of ssh configuration and merge all of them inside home user ~/.ssh/authorized_keys of the vm deployed. Signed-off-by: Eduardo Otubo <otubo@redhat.com>
2019-12-02url_helper: read_file_or_url should pass headers param into readurl (#66)Chad Smith
Headers param was accidentally omitted and no longer passed through to readurl due to a previous commit. To avoid this omission of params in the future, drop positional param definitions from read_file_or_url and pass all kwargs through to readurl when we are not operating on a file. In util:read_seeded, correct the case where invalid positional param file_retries was being passed into read_file_or_url. Also drop duplicated file:// prefix addition from read_seeded because read_file_or_url does that work anyway. LP: #1854084
2019-11-26cloud_tests: add azure platform support to integration testsahosmanmsft
Added Azure to cloud tests supporting upstream integration testing. Implement the inherited platform classes, Azure configurations to release/platform, and docs on how to run Azure CI.
2019-11-26set_passwords: support for FreeBSD (#46)Igor Galić
Allow setting of user passwords on FreeBSD The www/chpasswd utility which we depended on for FreeBSD installations does *not* do the same thing as the equally named Linux utility. For FreeBSD, we now use the pw(8) utility (which can only process one user at a time) Additionally, we abstract expire passwd into a function, and override it in the FreeBSD distro class. Co-Authored-By: Chad Smith <chad.smith@canonical.com>
2019-11-25FreeBSD: fix for get_linux_distro() and lru_cache (#59)Igor Galić
Since `is_FreeBSD()` is used a lot, which uses `system_info()`, which uses `get_linux_distro()` we add caching, by decorating the following functions with `@lru_cache`: - get_architecture() - _lsb_release() - is_FreeBSD - get_linux_distro - system_info() - _get_cmdline() Since [functools](https://docs.python.org/3/library/functools.html) only exists in Python 3, only python 3 will benefit from this improvement. For python 2, our shim is just a pass-thru. Too bad, but, also… https://pythonclock.org/ The main motivation here was, at first, to cache more, following the style of _lsb_release. That is now consolidated under this very same roof. LP: #1815030
2019-11-22ec2: Add support for AWS IMDS v2 (session-oriented) (#55)Ryan Harper
* ec2: Add support for AWS IMDS v2 (session-oriented) AWS now supports a new version of fetching Instance Metadata[1]. Update cloud-init's ec2 utility functions and update ec2 derived datasources accordingly. For DataSourceEc2 (versus ec2-look-alikes) cloud-init will issue the PUT request to obtain an API token for the maximum lifetime and then all subsequent interactions with the IMDS will include the token in the header. If the API token endpoint is unreachable on Ec2 platform, log a warning and fallback to using IMDS v1 and which does not use session tokens when communicating with the Instance metadata service. We handle read errors, typically seen if the IMDS is beyond one etwork hop (IMDSv2 responses have a ttl=1), by setting the api token to a disabled value and then using IMDSv1 paths. To support token-based headers, ec2_utils functions were updated to support custom headers_cb and exception_cb callback functions so Ec2 could store, or refresh API tokens in the event of token becoming stale. [1] https://docs.aws.amazon.com/AWSEC2/latest/ \ UserGuide/ec2-instance-metadata.html \ #instance-metadata-v2-how-it-works
2019-11-21tests: Fix cloudsigma tests when no dmidecode data is present. (#57)Scott Moser
The cloudsigma tests had few test cases that were not getting all the "mocks" set up correctly. Specifically is_running_in_cloudsigma was not getting replaced and calls would leak through to util.read_dmi_data.
2019-11-20net: IPv6, accept_ra, slaac, stateless (#51)Harald
Router advertisements are required for the default route to be set up, thus accept_ra should be enabled for dhcpv6-stateful. sysconf: IPV6_FORCE_ACCEPT_RA controls accept_ra sysctl. eni: mode static and mode dhcp 'accept_ra' controls sysctl. Add 'accept-ra: true|false' parameter to config v1 and v2. When True: accept_ra is set to '1'. When False: accept_ra is set to '0'. When not defined in config the value is left to the operating system default. This change also extend the IPv6 support to distinguish between slaac and dhcpv6-stateless. SLAAC is autoconfig without any options from DHCP, while stateless auto-configures the address and the uses DHCP for other options. LP: #1806014 LP: #1808647
2019-11-13azure: support secondary ipv6 addresses (#33)Chad Smith
Azure's Instance Metadata Service (IMDS) reports multiple IPv6 addresses, via the http://169.254.169.254/metadata/instance/network route. Any additional values after the first in 'ipAddresses' under the 'ipv6' interface key are extracted and configured as static IPs on the interface.
2019-11-08reporting: Using a uuid to enforce uniqueness on the KVP keys.momousta
The KVPs currently being emitted to the .kvp_pool file can have duplicate keys which is wrong since these keys should be unique. The situation can occur if for example one azure function called twice or more and this function is reporting telemetry through the use of KVPs. Any KVP consumer can get confused by the duplicate keys and a race condition can and have occurred.
2019-11-04azure: support matching dhcp route-metrics for dual-stack ipv4 ipv6Chad Smith
Network v2 configuration for Azure will set both dhcp4 and dhcp6 to False by default. When IPv6 privateIpAddresses are present for an interface in Azure's Instance Metadata Service (IMDS), set dhcp6: True and provide a route-metric value that will match the corresponding dhcp4 route-metric. The route-metric value will increase by 100 for each additional interface present to ensure the primary interface has a route to IMDS. Also fix dhcp route-metric rendering for eni and sysconfig distros. LP: #1850308
2019-11-04configdrive: fix subplatform config-drive for /config-drive sourceDavid Kindred
When ConfigDrive discovers the source path /config-drive, subplatform is now reports 'config-drive' LP: #1849731
2019-10-31DataSourceSmartOS: reconfigure network on each bootMike Gerdts
In typical cases, SmartOS does not use DHCP for network configuration. As such, if the network configuration changes that is reflected in metadata and will be picked up during the next boot. LP: #1765801 Joyent: OS-6902 reconfigure network on each boot
2019-10-31Add config for ssh-key import and consuming user-dataPavel Zakharov
This patch enables control over SSH public-key import and discarding supplied user-data (both disabled by default). allow-userdata: false ssh: allow_public_ssh_keys: false This feature enables closed appliances to prevent customers from unintentionally breaking the appliance which were not designed for user interaction. The downstream change for this is here: https://github.com/delphix/cloud-init/pull/4
2019-10-31net: fix subnet_is_ipv6() for stateless|statefulHarald Jensås
Function return false for ipv6_dhcpv6-stateless|stateful, the eni renderer does not add '6' to 'inet' which is incorrect. The subnet_is_ipv6() function is updated to also return true if startswith('ipv6'). LP: #1848690
2019-10-31OVF: disable custom script execution by defaultXiaofeng Wang
For security concern, we disable the custom script by default.If a custom script is provided, stop customization unless the custom script is explicitly enabled by tools config.
2019-10-31cc_puppet: Implement csr_attributes.yaml supportMatthias Baur
This change adds two new parameters: * csr_attributes * csr_attributes_path Those parameters allow to configure the content of the csr_attributes.yaml file. See https://puppet.com/docs/puppet/latest/config_file_csr_attributes.html
2019-10-29azure: Do not lock user on instance id changeSam Eiderman
After initial boot ovf-env.xml is copied to agent dir (/var/lib/waagent/) with REDACTED password. On subsequent boots DataSourceAzure loads with a configuration where the user specified in /var/lib/waagent/ovf-env.xml is locked. If instance id changes, cc_users_groups action will lock the user. Fix this behavior by not locking the user if its password is REDACTED. LP: #1849677
2019-10-24net/netplan: use ipv6-mtu key for specifying ipv6 mtu valuesRyan Harper
netplan introduced an 'info' subcommand which emits yaml describing implemented features that indicate new or changed fields and values in the yaml that it accepts. Previously, cloud-init emitted the key 'mtu6' for ipv6 MTU values. This is not correct and netplan will fail to parse these values. Netplan as of 0.98 supports both the info subcommand and the ipv6-mtu key. This branch modifies the netplan renderer to collect the netplan info output into a 'features' property which is a list of available feature flags which the renderer can use to modify its output. If the command is not available, no feature flags are set and cloud-init will render IPv6 MTU values just as MTU for the subnet.
2019-10-24Fix usages of yaml, and move yaml_dump to safeyaml.dumps.Scott Moser
Here we replace uses of the pyyaml module directly with functions provided by cloudinit.safeyaml. Also, change/move cloudinit.util.yaml_dumps to cloudinit.safeyaml.dumps LP: #1849640
2019-10-21net: enable infiniband support in eni and sysconfig renderersDarren Birkett
Commit e7b0e5f72 added support for configuring infiniband devices by adding a new infiniband 'type'. This commit updates eni and sysconfig renderers to consume this new type and configure infiniband devices correctly. LP: #1847114
2019-10-17guestcust_util: handle special characters in config fileXiaofeng Wang
Handle the special characters when reading VM Tools configure file. For example, the key and value may contain _, - and . etc.
2019-10-17replace any deprecated log.warn with log.warningDominic Schlegel
Commit 6797e822959b84c98cf73e02b2a6e3d6ab3fd4fe replaced the LOG.warn calls that linters were warning about; this also replaces calls that linters would not have recognised (as `log` is generally a parameter in these scenarios). LP: #1508442
2019-10-16net: handle openstack dhcpv6-stateless configurationHarald Jensås
Openstack subnets can be configured to use SLAAC by setting ipv6_address_mode=dhcpv6-stateless. When this is the case the sysconfig interface configuration should use IPV6_AUTOCONF=yes and not set DHCPV6C=yes. This change sets the subnets type property to the full network['type'] from openstack metadata. cloudinit/net/sysconfig.py and cloudinit/net/eni.py are updated to support new subnet types: - 'ipv6_dhcpv6-stateless' => IPV6_AUTOCONF=yes - 'ipv6_dhcpv6-stateful' => DHCPV6C=yes Type 'dhcp6' in sysconfig is kept for backward compatibility with any implementations that set subnet_type == 'dhcp6'. LP: #1847517
2019-10-11cloud_test/lxd: Retry container delete a few timesRyan Harper
LXD integration tests fail sometimes due to failure to delete the container, usually related to ZFS backend. This is a transient issue unrelated to the test itself. Teach LXD platform to retry this a few times before returning an error.
2019-10-11Add Support for e24cloud to Ec2 datasource.Scott Moser
e24cloud provides an EC2 compatible datasource. This just identifies their platform based on dmi 'system-vendor' having 'e24cloud'. https://www.e24cloud.com/en/ . Updated chassis typo in zstack unit test docstring. LP: #1696476
2019-10-04Add RbxCloud datasourceAdam Dobrawy
2019-09-27debian/ubuntu: add missing word to netplan/ENI headerDaniel Watkins
Specifically, add in "reboot" to make it clear what people should expect when modifying the file. This also renames the variable to indicate it is used for netplan and ENI, not just ENI. LP: #1845669
2019-09-27ovf: do not generate random instance-id for IMC customization pathRyan Harper
Cloud-init will not operate properly if the instance-id value changes on each boot. This is the source of a number of behavioral bugs filed against cloud-init with OVF datasource. Instead, use a static instance-id value, iid-vmware-imc, similar to iid-dsovf.
2019-09-26sysconfig: only write resolv.conf if network_state has DNS valuesRyan Harper
If an OS image provided an /etc/resolv.conf file that was not empty cloud-init would read and re-write it with a cloud-init header even if no DNS network configuration was provided (e.g. DHCP only). This can cause problems for some network services which don't ignore cloud-init's header. LP: #1843634
2019-09-26sysconfig: use distro variant to check if availableRyan Harper
The sysconfig renderer used the distro name directly which mean some variants of distros were not considered supported. Fix this by using util.system_info()['variant'] instead. Fix the list of KNOWN_DISTROS value for redhat -> rhel. LP: #1843584
2019-09-18Add datasource for ZStack platform.Shixin Ruan
Zstack platform provides a AWS Ec2 metadata service, and identifies their platform to the guest by setting the 'chassis asset tag' to a string that ends with '.zstack.io'. LP: #1841181
2019-09-11Brightbox: restrict detection to require full domain match .brightbox.comScott Moser
The detection for brightbox in both ds-identify and in identify_brightbox would incorrectly match the domain 'bobrightbox', which is not a brightbox platform. The fix here is to restrict matching to '*.brightbox.com' rather than '*brightbox.com' Also, while here remove a url to bug 1661693 which added the knowledge of brightbox.
2019-09-11VMWware: add option into VMTools config to enable/disable custom script.Xiaofeng Wang
VMWware customization already has support to run a custom script during the VM customization. Adding this option allows a VM administrator to disable the execution of customization scripts. If set the script will not execute and the customization status is set to GUESTCUST_ERROR_SCRIPT_DISABLED.
2019-08-28exoscale: fix sysconfig cloud_config_modules overridesChad Smith
Make sure Exoscale supplements or overrides existing system config setting cloud_config_modules instead of replacing it with a one item list set-passords LP: #1841454
2019-08-22net/cmdline: refactor to allow multiple initramfs network config sourcesDaniel Watkins
This refactors read_initramfs_config to support multiple different types of initramfs network configuration. It introduces an InitramfsNetworkConfigSource abstract base class. There is currently a single sub-class, KlibcNetworkConfigSource, which contains the logic which previously was directly within read_initramfs_config.
2019-08-19ubuntu-drivers: emit latelink=true debconf to accept nvidia eulaChad Smith
To accept NVIDIA EULA, cloud-init needs to emit latelink=true debconf setting to the linux-restricted-modules package to allow NVIDIA drivers to properly link to the running kernel. LP: #1840080
2019-08-14Azure: Record boot timestamps, system information, and diagnostic eventsAnh Vo
Collect and record the following information through KVP:  + timestamps related to kernel initialization and systemd activation    of cloud-init services  + system information including cloud-init version, kernel version,    distro version, and python version  + diagnostic events for the most common provisioning error issues    such as empty dhcp lease, corrupted ovf-env.xml, etc. + increasing the log frequency of polling IMDS during reprovision.
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-10 13:44:54 +0000