vyos-cloud-init.git/tests/unittests/test_datasource/test_ec2.py, branch rolling

Reorganize unit test locations under tests/unittests (#1126)

2021-12-03T20:11:46+00:00

This attempts to standardize unit test file location under test/unittests/ such that any source file located at cloudinit/path/to/file.py may have a corresponding unit test file at test/unittests/path/to/test_file.py. Noteworthy Comments: ==================== Four different duplicate test files existed: test_{gpg,util,cc_mounts,cc_resolv_conf}.py Each of these duplicate file pairs has been merged together. This is a break in git history for these files. The test suite appears to have a dependency on test order. Changing test order causes some tests to fail. This should be rectified, but for now some tests have been modified in tests/unittests/config/test_set_passwords.py. A helper class name starts with "Test" which causes pytest to try executing it as a test case, which then throws warnings "due to Class having __init__()". Silence by changing the name of the class. # helpers.py is imported in many test files, import paths change cloudinit/tests/helpers.py -> tests/unittests/helpers.py # Move directories: cloudinit/distros/tests -> tests/unittests/distros cloudinit/cmd/devel/tests -> tests/unittests/cmd/devel cloudinit/cmd/tests -> tests/unittests/cmd/ cloudinit/sources/helpers/tests -> tests/unittests/sources/helpers cloudinit/sources/tests -> tests/unittests/sources cloudinit/net/tests -> tests/unittests/net cloudinit/config/tests -> tests/unittests/config cloudinit/analyze/tests/ -> tests/unittests/analyze/ # Standardize tests already in tests/unittests/ test_datasource -> sources test_distros -> distros test_vmware -> sources/vmware test_handler -> config # this contains cloudconfig module tests test_runs -> runs

printing the error stream of the dhclient process before killing it (#369)

2020-06-19T13:38:05+00:00

This introduces a way to log the dhclient error stream, and uses it for the Azure datasource (where we have a specific requirement for this data to be logged).

test: fix all flake8 E121 and E123 errors (#404)

2020-06-02T16:06:07+00:00

This fixes issues with closing brackets not matching the opening bracket's line and continuation line under-idented for hanging indent.

test: fix all flake8 E741 errors (#401)

2020-06-01T21:20:39+00:00

This removes the use of variables named ‘l’, ‘O’, or ‘I’. Generally these are used in list comprehension to read the line of lines.

DataSourceEc2: use metadata's NIC ordering to determine route-metrics (#342)

2020-05-01T20:56:56+00:00

We want to set route-metrics such that NICs are configured with the priority that they are given in the network metadata that we receive from the IMDS. (This switches away from using MAC ordering.) This also required the following test changes: * reverse the sort order of the MACs in test data (so that they would trigger the bug being fixed) * fix up the key names in `NIC2_MD` (which were under_scored instead of dash-separated) * use a full interface dict (rather than a minimal one) for `TestConvertEc2MetadataNetworkConfig` LP: #1876312

ec2: render network on all NICs and add secondary IPs as static (#114)

2020-03-18T19:33:37+00:00

Add support for rendering secondary static IPv4/IPv6 addresses on any NIC attached to the machine. In order to see secondary IP addresses in Ec2 IMDS network config, cloud-init now reads metadata version 2018-09-24. Metadata services which do not support the Ec2 API version will not get secondary IP addresses configured. In order to discover secondary IP address config, cloud-init now relies on metadata API Parse local-ipv4s, ipv6s, subnet-ipv4-cidr-block and subnet-ipv6-cidr-block metadata keys to determine additional IPs and appropriate subnet prefix to set for a nic. Also add the datasource config option apply_full_imds_netork_config which defaults to true to allow cloud-init to automatically configure secondary IP addresses. Setting this option to false will tell cloud-init to avoid setting up secondary IP addresses. Also in this branch: - Shift Ec2 datasource to emit network config v2 instead of v1. LP: #1866930

ec2: Do not fallback to IMDSv1 on EC2 (#216)

2020-03-06T00:38:28+00:00

The EC2 Data Source needs to handle 3 states of the Instance Metadata Service configured for a given instance: 1. HttpTokens : optional & HttpEndpoint : enabled Either IMDSv2 or IMDSv1 can be used. 2. HttpTokens : required & HttpEndpoint : enabled Calls to IMDS without a valid token (IMDSv1 or IMDSv2 with expired token) will return a 401 error. 3. HttpEndpoint : disabled The IMDS http endpoint will return a 403 error. Previous work to support IMDSv2 in cloud-init handled case 1 and case 2. This commit handles case 3 by bypassing the retry block when IMDS returns HTTP status code >= 400 on official AWS cloud platform. It shaves 2 minutes when rebooting an instance that has its IMDS http token endpoint disabled but creates some inconsistencies. An instance that doesn't set "manual_cache_clean" to "True" will have its /var/lib/cloud/instance symlink removed altogether after it has failed to find a datasource.

ec2: Do not log IMDSv2 token values, instead use REDACTED (#219)

2020-02-19T21:01:09+00:00

Instead of logging the token values used log the headers and replace the actual values with the string 'REDACTED'. This allows users to examine cloud-init.log and see that the IMDSv2 token header is being used but avoids leaving the value used in the log file itself. LP: #1863943

Replace mock library with unittest.mock (#186)

2020-01-29T21:55:39+00:00

* cloudinit: replace "import mock" with "from unittest import mock" * test-requirements.txt: drop mock Co-authored-by: Chad Smith

ec2: Add support for AWS IMDS v2 (session-oriented) (#55)

2019-11-23T03:05:44+00:00

* ec2: Add support for AWS IMDS v2 (session-oriented) AWS now supports a new version of fetching Instance Metadata[1]. Update cloud-init's ec2 utility functions and update ec2 derived datasources accordingly. For DataSourceEc2 (versus ec2-look-alikes) cloud-init will issue the PUT request to obtain an API token for the maximum lifetime and then all subsequent interactions with the IMDS will include the token in the header. If the API token endpoint is unreachable on Ec2 platform, log a warning and fallback to using IMDS v1 and which does not use session tokens when communicating with the Instance metadata service. We handle read errors, typically seen if the IMDS is beyond one etwork hop (IMDSv2 responses have a ttl=1), by setting the api token to a disabled value and then using IMDSv1 paths. To support token-based headers, ec2_utils functions were updated to support custom headers_cb and exception_cb callback functions so Ec2 could store, or refresh API tokens in the event of token becoming stale. [1] https://docs.aws.amazon.com/AWSEC2/latest/ \ UserGuide/ec2-instance-metadata.html \ #instance-metadata-v2-how-it-works