summaryrefslogtreecommitdiff
path: root/tests/unittests
AgeCommit message (Collapse)Author
2022-02-15sources/azure: report ready in local phase (#1265)Chris Patterson
Pre-provisioned instances report ready early in the local phase and again in the non-local phase, during setup(). Non-PPS only reports ready during non-local phase. Update the process to report ready during the local phase for all cases. Only attempt to do so if networking is up to prevent stalling boot. We've already waited at least 20 minutes for DHCP if we're provisioning, or 5 minutes for DHCP on normal boot requesting updated network configuration. - Extend _report_ready() with pubkey_info and raise exception on error to consolidate reporting done in _negotiate() and _report_ready(). - Remove setup(), moving relevant logic into crawl_metadata(). - Move remaining _negotiate() logic into _cleanup_markers() and _determine_wireserver_pubkey_info(). These changes effectively fix two issues that were present: (1) _negotiated is incorrectly set to True When failing to report ready. _negotiate() squashed the exception and the return value was not checked. This was probably masked due to the forced removal of obj.pkl on Ubuntu instances, but would be preferable once we start persisting it to prevent unnecessary re-negotiation. (2) provisioning media is not ejected for non-PPS _negotiate() did not pass iso_dev parameter when reporting ready. The host will ensure this operation takes place, but it is preferable to eject /dev/sr0 from within the guest when we're done with it. Lastly, this removes any need for lease file parsing as the wireserver addressed is tracked for ephemeral DHCP. A follow-up PR will remove this now-unused logic. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
2022-02-14sources/azure: validate IMDS network configuration metadata (#1257)Chris Patterson
Due to race conditions and caching, IMDS may return stale or incomplete metadata. Add some validation to detect these scenarios and report appropriate telemetry. Introduce normalize_mac_address() to allow for comparison of mac addresses, replacing that found inline in: _generate_network_config_from_imds_metadata() Add validation of final fetch of IMDS metadata. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
2022-02-14use PEP 589 syntax for TypeDict (#1253)Brett Holman
Use PEP 589 syntax for TypeDict annotation. Also fixes previously broken typing MetaSchema typing implementation.
2022-02-14Fix extra ipv6 issues, code reduction and simplification (#1243)eb3095
Eliminated the duplicate code and now run the entire configuration routine against both public and private interfaces. Also addressed an inconsistency from our metadata api for ipv6 address configuration.
2022-02-11sources/azure: address mypy/pyright typing complaints (#1245)Chris Patterson
Raise runtime errors for unhandled cases which would cause other exceptions. Ignore types for a few cases where a non-trivial refactor would be required to prevent the warning. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
2022-02-10Shell script handlers by freq (#1166)Chris Lalos
Handlers for per-boot/per-instance/per-once multipart MIME Add handlers for adding scripts to userdata that can be run at various frequencies. Scripts of type x-shellscript-per-boot, x-shellscript-per-instance, or x-shellscript-per-once can be added to a multipart MIME userdata message as part of instance userdata. These scripts will then be added to the appropriate per-boot, per-instance, or per-once directory in /var/lib/cloud/scripts/ during processing of userdata.
2022-02-10cloud-id: publish /run/cloud-init/cloud-id-<cloud-type> files (#1244)Chad Smith
Once a valid datasource is detected, publish the following artifacts to expedite cloud-identification without having to invoke cloud-id from shell scripts or sheling out from python. These files can also be relied on in systemd ConditionPathExists directives to limit execution of services and units to specific clouds. /run/cloud-init/cloud-id: - A symlink with content that is the canonical cloud-id of the datasource detected. This content is the same lower-case value as the output of /usr/bin/cloud-id. /run/cloud-init/cloud-id-<canonical-cloud-id>: - A single file which will contain the canonical cloud-id encoded in the filename
2022-02-10sources/azure: refactor ssh key handling (#1248)Chris Patterson
Split _get_public_ssh_keys_and_source() into _get_public_keys_from_imds() and _get_public_keys_from_ovf(). Set _get_public_keys_from_imds() to take a parameter of the IMDS metadata rather than assuming it is already set in self.metadata. This will allow us to move negotation into local phase where self.metadata may not be set yet. Update this method to raise KeyError if IMDS metadata is missing/malformed, and ValueError if SSH key format is not supported. Update get_public_ssh_keys() to catch these errors and fall back to the OVF/Wireserver keys as needed. To improve clarity, update register_with_azure_and_fetch_data() to return the list of SSH keys, rather than bundling them into a dictionary for updating against the metadata dictionary. There should be no change in behavior with this refactor. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
2022-02-09sources/hetzner: Use EphemeralDHCPv4 instead of static configuration (#1251)Markus Schade
When the datasource was originally submitted, EphemeralDHCPv4 was not yet available. Also avoid race conditions by skipping network configuration if metadata service can be reached. Signed-off-by: Markus Schade <markus.schade@hetzner.com>
2022-02-08Fix IPv6 netmask format for sysconfig (#1215)Harald
This change converts the IPv6 netmask from the network_data.json[1] format to the CIDR style, <IPv6_addr>/<prefix>. Using an IPv6 address like ffff:ffff:ffff:ffff:: does not work with NetworkManager, nor networkscripts. NetworkManager will ignore the route, logging: ifcfg-rh: ignoring invalid route at \ "::/:: via fd00:fd00:fd00:2::fffe dev $DEV" \ (/etc/sysconfig/network-scripts/route6-$DEV:3): \ Argument for "::/::" is not ADDR/PREFIX format Similarly if using networkscripts, ip route fail with error: Error: inet6 prefix is expected rather than \ "fd00:fd00:fd00::/ffff:ffff:ffff:ffff::". Also a bit of refactoring ... cloudinit.net.sysconfig.Route.to_string: * Move a couple of lines around to reduce repeated code. * if "ADDRESS" not in key -> continute, so that the code block following it can be de-indented. cloudinit.net.network_state: * Refactors the ipv4_mask_to_net_prefix, ipv6_mask_to_net_prefix removes mask_to_net_prefix methods. Utilize ipaddress library to do some of the heavy lifting. LP: #1959148
2022-02-04sources/azure: consolidate ephemeral DHCP configuration (#1229)Chris Patterson
Introduce: - _setup_ephemeral_networking() to bring up networking. If no iface is specified, it will use net.find_fallback_nic() which is consistent with the previous usage of fallback_interface. This method now tracks the encoded address of the wireserver with a new property `_wireserver_endpoint`. Introduce a timeout parameter to allow for retrying for a specified amount of time. - _teardown_ephemeral_networking() to bring down networking. - _is_ephemeral_networking_up() to check status. Ephemeral networking is now: - Brought up prior to checking IMDS. - Torn down following metadata crawl. - For Savable PPS, torn down prior to waiting for NIC detach. The link must be torn down in advance or we will see errors from cleaning up network after the interface is unplugged. - For Running PPS, torn down after waiting for media switch. The link must be up for media switch to be detected. - For all PPS, after network switch is complete, networking is brought back up to poll for reprovision data and report ready. It will be torn down after metadata crawl is complete like non-PPS paths. Additionally: - Remove EphemeralDHCPv4WithReporting variant in favor of directly using EphemeralDHCPv4. The reporting was only for __enter__ usage which is no longer a used path. Continue to use dhcp_log_cb callback. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
2022-02-04sources/azure: fix metadata check in _check_if_nic_is_primary() (#1232)Chris Patterson
Currently _check_if_nic_is_primary() checks for imds_md is None, but imds_md is returned as an empty dictionary on error fetching metdata. Fix this check and the tests that are incorrectly vetting IMDS polling code. Additionally, use response.contents instead of str(response) when loding the JSON data returned from readurl. This helps simplify the mocking and avoids an unncessary conversion. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
2022-01-31Schema a d (#1211)Chad Smith
Migrate from legacy schema or define new schema in cloud-init-schema.json, adding extensive schema tests for: - cc_apt_configure - cc_bootcmd - cc_byobu - cc_ca_certs - cc_chef - cc_debug - cc_disable_ec2_metadata - cc_disk_setup Deprecate config hyphenated schema keys in favor of underscores: - ca_certs and ca_certs.remove_defaults instead of ca-certs and ca-certs.remove-defaults - Continue to honor deprecated config keys but emit DEPRECATION warnings in logs for continued use of the deprecated keys: - apt_sources key - any apt v1 or v2 keys - use or ca-certs or ca_certs.remove-defaults - Extend apt_configure schema - Define more strict schema below object opaque keys using patternProperties - create common $def apt_configure.mirror for reuse in 'primary' and 'security' schema definitions within cc_apt_configure Co-Authored-by: James Falcon <james.falcon@canonical.com>
2022-01-31sources/azure: refactor _report_ready_if_needed and _poll_imds (#1222)Chris Patterson
Refactor _report_ready_if_needed() to work for both Savable PPS and Runnable PPS: * rename _report_ready_if_needed() to _report_ready_for_pps() * return interface name from lease to support _poll_imds() behavior without changing it. * fixes an issue where reporting ready return value was silently ignored for Savable PPS. * add explicit handling for failure to obtain DHCP lease to result in sources.InvalidMetaDataException. Refactor _poll_imds(): * use _report_ready_for_pps() for reporting ready, removing this logic to simplify loop logic. * move netlink and vnetswitch out of while loop to simplify loop logic, leaving only reprovision polling in loop. * add explicit handling for failure to obtain DHCP lease and retry in the next iteration. Signed-off-by: Chris Patterson cpatterson@microsoft.com
2022-01-28sources/azure: refactor _should_reprovision[_after_nic_attach]() logic (#1206)Chris Patterson
Consolidate _should_reprovision_after_nic_attach() with _should_reprovision() into the following: _write_reprovision_marker() to write provisioning marker for reboot-during-provisioning case. PPSType enum and _determine_pps_type() for determining which to provisioning mode, if any, we're running under. PPSType.UNKNOWN is when the reprovisioning marker is found and we do not have the context to know what the original mode was. In this scenario, we must resort to polling for reprovision data. Tests: Introduce a simple data source fixture to for fine-grain control of mocking with pytest without unittest. Migrate relevant _should_reprovision() tests into a combination of TestDeterminePPSTypeScenarios cases. Signed-off-by: Chris Patterson cpatterson@microsoft.com
2022-01-26Add json parsing of ip addr show (SC-723) (#1210)James Falcon
When obtaining information from "ip addr", default to using "ip --json addr" rather than using regex to parse "ip addr show" as json is machine readable as less prone to error. Deprecate but leave fallback to use "ip addr" for older iproute2 tooling which does not support --json param. Fix regex parsing of "ip addr" to support peer addresses and metrics.
2022-01-20sources/azure: drop unused case in _report_failure() (#1200)Chris Patterson
According to the documentation in the tests: ``` We expect 3 calls to report_failure_to_fabric, because we try 3 different methods of calling report failure. The different methods are attempted in the following order: 1. Using cached ephemeral dhcp context to report failure to Azure 2. Using new ephemeral dhcp to report failure to Azure 3. Using fallback lease to report failure to Azure ``` Case 1 and 2 make sense. If networking is established, use it. Should failure occur using current network configuration, retry with fresh DHCP. Case 3 suggests that we can fall back to a lease file and retry. Given that: 1. The wireserver address has never changed to date. 2. The wireserver address should be in the DHCP lease. 3. Parsing the lease file does not improve connectivity over the prior attempts. ...we can safely remove this case without regression. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
2022-01-20Add support for gentoo templates and cloud.cfg (#1179)vteratipally
2022-01-20sources/azure: unpack ret tuple in crawl_metadata() (#1194)Chris Patterson
load_azure_ds_dir() always returns a tuple. Instead of saving this tuple as ret, expand it immediately as md, userdata_raw, cfg, files. This allows for more fine-grained passing of data before getting expanded later. - Update _should_reprovision methods to use cfg instead of tuple. - Update _should_reprovision methods to remove the ovf_md guard. This should be a safe refactor as the OVF is not required, and the config is initialized to an empty dict. In practice, a mount failure would have initialized ret anyways if the OVF was not found. If a mount failure wasn't seen and ret was None, this guard could be causing other failures by ignoring the PPS state that should be available from IMDS metadata. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
2022-01-19tests: focal caplog has whitespace indentation for multi-line logs (#1201)Chad Smith
Avoid series-specific log formatting in tests by using caplog.record_tuples. Added benefit, we can easily check log-level WARNING too. Fixes unit tests from Focal Package builds at: https://code.launchpad.net/~cloud-init-dev/+archive/ubuntu/daily/\ +build/23076508
2022-01-18cmd: status and cloud-id avoid change in behavior for 'not run' (#1197)Chad Smith
snapd currrently looks for 'not run' from cloud-init status[1]. Avoid changing this behavior and revert "not-run" value to "not run". This avoids having to get snapd to change implementation and release updates as far back as Bionic to handle a hyphenated not-run string. [1]: https://github.com/snapcore/snapd/blob/master/sysconfig/\ cloudinit.go#L802
2022-01-18sources/azure: set ovf_is_accessible when OVF is read successfully (#1193)Chris Patterson
The if-statement set ovf_is_accessible to True if the OVF is read from /dev/sr0, but not from other data sources. It defaults to True, but may get flipped to False while processing an invalid source, and never get set back to True when reading from the data directory. Instead, default ovf_is_accessible to False, and only set it to True once we've read an OVF successfully (and end the search). This fixes an error when OVF is read from data_dir and IMDS data is unavailable (failing with "No OVF or IMDS available"). Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
2022-01-18sources/azure: consolidate DHCP variants to EphemeralDHCPv4WithReporting (#1190)Chris Patterson
- Update EphemeralDHCPv4WithReporting to subclass EphemeralDHCPv4 for consistency (non-functional change). - Replace all usage of EphemeralDHCPv4 with EphemeralDHCPv4WithReporting. - Converging to one DHCP class exposed an issue with ExitStack patches being mixed with decorators. Specifically, it appeared that tests that did not enable azure.EphemeralDHCPv4WithReporting mocks had it applied anyways from previous tests. Presumably ExitStack was overwriting the actual value with the mock provided by the decorator? For now, remove some mock patches that trigger failures, but future work should move towards a consistent approach to prevent undetected effects. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
2022-01-18Single JSON schema validation in early boot (#1175)Chad Smith
Package a single JSON schema file for user-data validation at cloudinit/config/cloud-init-schema.json. Perform validate_cloudconfig_schema call to just after the user-data is consumed. This will allow single validation of all user-data against the full schema instead of repetitive validatation calls against each cloud-config module (cloudinit.config.cc_*) sub-schemas. This branch defines the simple apt_pipelining schema and migrates existing cc_apk_configure into cloud-init-schema.json. The expectation will be additional branches to migrate from legacy "schema" attributes inside each cloud-config module toward unique cc_<module_name> definitions in the global shema file under "$defs" of cloud-init-schema-X.Y..json. Before legacy sub-schema definitions are migrated the following funcs grew support to read sub-schemas from both static cloud-init-schema.json and the individual cloud-config module "schema" attributes: - get_schema: source base schema file from cloud-init-schema.json and supplement with all legacy cloud-config module "schema" defs - get_meta_doc: optional schema param so cloud-config modules no longer provide the own local sub-schemas - _get_property_doc: render only documentation of sub-schema based on meta['id'] provided - validate_cloudconfig_schema: allow optional schema param Additionally, fix two minor bugs in _schemapath_for_cloudconfig: - `cloud-init devel schema --annotate` which results in a Traceback if two keys at the same indent level have invalid types. - exit early on empty cloud-config to avoid a Traceback on the CLI
2022-01-15Ensure system_cfg read before ds net config on Oracle (SC-720) (#1174)James Falcon
In 2c52e6e88b19f5db8d55eb7280ee27703e05d75f, the order of reading network config was changed for Oracle due to initramfs needing to take lower precedence than the datasource. However, this also bumped system_cfg to a lower precedence than ds, which means that any network configuration specified in /etc/cloud will not be applied. system_cfg should instead be moved above ds so network configuration in /etc/cloud takes precedence. LP: #1956788
2022-01-14Test Optimization Proposal (SC-736) (#1188)Brett Holman
Reduce template rendering test runtime
2022-01-13cli: cloud-id report not-run or disabled state as cloud-id (#1162)Chad Smith
This fix has two elements: - cloud-init status will not correctly report 'not-run' prior to systemd generator running. Only report "disabled" when generator has run and /run/cloud-init/disabled exists. - Expose not-run and disabled state in cloud-id responses - Add unique error codes from cloud-id for error, disabled and not-run. The new cloud-id exit codes: 0: success 1: error 2: cloud-init is in disabled state 3: cloud-init generator has not run yet
2022-01-12Add new config module to set keyboard layout (#1176)maxnet
Adds a new module to allow setting keyboard layout, for use-cases in which cloud-init is used to configure OS images meant for physical computers instead of the cloud. This initial release only implements support for Linux distributions that allow layout to be set through systemd's localectl. LP: #1951593
2022-01-11sources/azure: rename metadata_type -> MetadataType (#1181)Chris Patterson
Format tweak to match naming conventions for classes & enums. No functional changes. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
2022-01-10Remove 3.5 and xenial support (SC-711) (#1167)James Falcon
Includes: - Update tox.ini and .travis.yml accordingly - Cleanup tox.ini with new tox syntax and cloud-init dependencies - Update documentation accordingly - Replace/remove xenial references where additional testing isn't required - Remove xenial checks in integration tests - Replace yield_fixture with fixture in pytest tests Sections of code commented with lines like "Remove when Xenial is no longer supported" still exist as they're require additional testing.
2022-01-10tests: mock LXD datasource detection in ds-identify on LXD containers (#1178)Chad Smith
On LXD containers /dev/lxd/sock will always exist. Mock dscheck_LXD to return 1 (NOT_FOUND) to avoid leaking into test environment and returning LXD as detected. We have integration tests covering proper LXD datasource detection so we don't need a Unit test validating the [ -S /dev/lxd/sock] that is in ds-identify.
2022-01-06Don't throw exceptions for empty cloud config (#1130)Brett Holman
Warn during boot when an empty config is provided. Likewise, `cloud-init devel schema --annotate` should not throw exception, return something meaningful instead.
2022-01-04sources/azure: do not persist failed_desired_api_version flag (#1159)Chris Patterson
If get_imds_data_with_api_fallback() falls back to the minimum required API version, it is effectively pinned to the old API version forever. Remove the failed_desired_api_version property to prevent persistence of the flag between calls and/or reboots. The continued presence of this flag in obj.pkl should be harmless. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
2022-01-04Update cc_ubuntu_advantage calls to assume-yes (#1158)John Chittum
cloud-init currently makes calls to ubuntu_advantage without assume-yes. some ua enable commands, such as ua enable fips, have prompts. In an automated environment, calling ua enable without --assume-yes will result in errors and not applying the change. This sets --assume-yes by default for all enable commands. This capability was added two years ago in ua commit 576e605ceb5f so should be safe for use in all systems at this time. LP: #1954842
2021-12-15Adopt Black and isort (SC-700) (#1157)James Falcon
Applied Black and isort, fixed any linting issues, updated tox.ini and CI.
2021-12-14tests/cmd/query: fix test run as root and add coverage for defaults (#1156)Chris Patterson
test_handle_args_error_on_invalid_vaname_paths() would fail when run as root due to invocation of load_userdata() on the default user/vendor data locations under the instance link. - Mock load_userdata() for this test case to avoid loads. - Update _setup_paths() to configure cloud_dir in temporary location. - Add new test case to verify that the default locations are loaded when unspecified. LP: #1825027 Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
2021-12-13Schema processing changes (SC-676) (#1144)James Falcon
* Use proper logging * Add parsing for patternProperties * Add label to annotate patternProperties * Log warning if schema parsing fails during metaschema processing * Some schema test fixes
2021-12-13find_devs_with_openbsd: ensure we return the last entry (#1149)Gonéri Le Bouder
`sysctl -n hw.disknames` returns a trailing `\n`. We need to clean this up. In addition, the criteria matching system is a source of problem because: - we don't have a way to look up the label of the partition - we've got situation where an ISO image can be exposed through a virtio block device. So we just totally ignore the value of `criteria`. We end-up with a slightly longer loop of mount-retry. But this way we're sure we don't miss a configuration disk. Tested on Kubvirt with the help of Brady Pratt @jbpratt.
2021-12-10sources/azure: remove unnecessary hostname bounce (#1143)Chris Patterson
Thanks to [1], the hostname is set prior to network bring-up. The Azure data source has been bouncing the hostname during setup(), occurring after the hostname has already been properly configured. Note that this doesn't prevent leaking the image's hostname during Azure's _get_data() when it brings up ephemeral DHCP. However, as are not guaranteed to have the hostname metadata available from a truly "local" source, this behavior is to be expected unless we disable `send host-name` from dhclient config. [1]: https://github.com/canonical/cloud-init/commit/133ad2cb327ad17b7b81319fac8f9f14577c04df Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
2021-12-09find_devs/openbsd: accept ISO on disk (#1132)Gonéri Le Bouder
When the metadata is an ISO image and is exposed through a disk, the device is called `/dev/sd?a` internally. For instance `/dev/sd1a`. It can then be mounted with `mount_cd9660 /dev/sd1a /mnt`. Metadata in the FAT32 format are exposed as `/dev/sd?i`. With this change, we try to mount `/dev/sd?a` in addition to `/dev/sd?i`. Closes: https://github.com/ContainerCraft/kmi/issues/12
2021-12-08factor out function for getting top level directory of cloudinit (#1136)Brett Holman
Add a test helper to get top level directory Many tests need to get the location of files & dirs within the cloud-init project directory. Tests implement this in various different ways, and often those ways depend on the current working directory of the pytest invocation. Create helper functions (and tests) that gets the path of the top directory or any sub directory under the top directory. This function does not depend on the environment.
2021-12-07testing: Add deterministic test id (#1138)Brett Holman
Parametrized pytest tests get named by on their parameters. If a name has random characters, it can break the test collection of when using pytest-xdist. Replace random name with deterministic name.
2021-12-07mock sleep() in azure test (#1137)Brett Holman
2021-12-07Add miraclelinux support (#1128)Haruki TSURUMOTO
2021-12-06Add Strict Metaschema Validation (#1101)Brett Holman
Improve schema validation. This adds strict validation of config module definitions at testing time, with plumbing included for future runtime validation. This eliminates a class of bugs resulting from schemas that have definitions that are incorrect, but get interpreted by jsonschema as "additionalProperties" that are therefore ignored. - Add strict meta-schema for jsonschema unit test validation - Separate schema from module metadata structure - Improve type annotations for various functions and data types Cleanup: - Remove unused jsonschema "required" elements - Eliminate manual memoization in schema.py:get_schema(), reference module.__doc__ directly
2021-12-06cloudinit/net: handle two different routes for the same ip (#1124)Emanuele Giuseppe Esposito
If we set a dhcp server side like this: $ cat /var/tmp/cloud-init/cloud-init-dhcp-f0rie5tm/dhcp.leases lease { ... option classless-static-routes 31.169.254.169.254 0.0.0.0,31.169.254.169.254 10.112.143.127,22.10.112.140 0.0.0.0,0 10.112.140.1; ... } cloud-init fails to configure the routes via 'ip route add' because to there are two different routes for 169.254.169.254: $ ip -4 route add 192.168.1.1/32 via 0.0.0.0 dev eth0 $ ip -4 route add 192.168.1.1/32 via 10.112.140.248 dev eth0 But NetworkManager can handle such scenario successfully as it uses "ip route append". So change cloud-init to also use "ip route append" to fix the issue: $ ip -4 route append 192.168.1.1/32 via 0.0.0.0 dev eth0 $ ip -4 route append 192.168.1.1/32 via 10.112.140.248 dev eth0 Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> RHBZ: #2003231
2021-12-03Reorganize unit test locations under tests/unittests (#1126)Brett Holman
This attempts to standardize unit test file location under test/unittests/ such that any source file located at cloudinit/path/to/file.py may have a corresponding unit test file at test/unittests/path/to/test_file.py. Noteworthy Comments: ==================== Four different duplicate test files existed: test_{gpg,util,cc_mounts,cc_resolv_conf}.py Each of these duplicate file pairs has been merged together. This is a break in git history for these files. The test suite appears to have a dependency on test order. Changing test order causes some tests to fail. This should be rectified, but for now some tests have been modified in tests/unittests/config/test_set_passwords.py. A helper class name starts with "Test" which causes pytest to try executing it as a test case, which then throws warnings "due to Class having __init__()". Silence by changing the name of the class. # helpers.py is imported in many test files, import paths change cloudinit/tests/helpers.py -> tests/unittests/helpers.py # Move directories: cloudinit/distros/tests -> tests/unittests/distros cloudinit/cmd/devel/tests -> tests/unittests/cmd/devel cloudinit/cmd/tests -> tests/unittests/cmd/ cloudinit/sources/helpers/tests -> tests/unittests/sources/helpers cloudinit/sources/tests -> tests/unittests/sources cloudinit/net/tests -> tests/unittests/net cloudinit/config/tests -> tests/unittests/config cloudinit/analyze/tests/ -> tests/unittests/analyze/ # Standardize tests already in tests/unittests/ test_datasource -> sources test_distros -> distros test_vmware -> sources/vmware test_handler -> config # this contains cloudconfig module tests test_runs -> runs
2021-12-02Fix exception when no activator found (#1129)James Falcon
Given that there are additional network management tools that we haven't yet supported with activators, we should log a warning and continue without network activation here, especially since this was a no-op for years. LP: #1948681
2021-12-02jinja: provide and document jinja-safe key aliases in instance-data (SC-622) ↵Chad Smith
(#1123) Allow #cloud-config and cloud-init query to use underscore-delimited "jinja-safe" key aliases for any instance-data.json keys containing jinja operator characters. This provides a means to use Jinja's dot-notation instead of square brackets and quoting to reference "unsafe" obtain attribute names. Support for these aliased keys is available to both #cloud-config user-data and `cloud-init query`. For example #cloud-config alias access can look like: {{ ds.config.user_network_config }} - instead of - {{ ds.config["user.network-config"] }}
2021-12-02Move GCE metadata fetch to init-local (SC-502) (#1122)James Falcon
GCE currently fetches metadata after network has come up. There's no reason we can't fetch at init-local time, so update GCE to fetch at init-local time to be more performant and consistent with other datasources.