| Age | Commit message (Collapse) | Author |
|---|
|
LP: #1876941
|
|
Improve schema validation.
This adds strict validation of config module definitions at testing
time, with plumbing included for future runtime validation. This
eliminates a class of bugs resulting from schemas that have definitions
that are incorrect, but get interpreted by jsonschema as
"additionalProperties" that are therefore ignored.
- Add strict meta-schema for jsonschema unit test validation
- Separate schema from module metadata structure
- Improve type annotations for various functions and data types
Cleanup:
- Remove unused jsonschema "required" elements
- Eliminate manual memoization in schema.py:get_schema(),
reference module.__doc__ directly
|
|
Fix dead link to "contributing" page in README
|
|
If we set a dhcp server side like this:
$ cat /var/tmp/cloud-init/cloud-init-dhcp-f0rie5tm/dhcp.leases
lease {
...
option classless-static-routes 31.169.254.169.254 0.0.0.0,31.169.254.169.254
10.112.143.127,22.10.112.140 0.0.0.0,0 10.112.140.1;
...
}
cloud-init fails to configure the routes via 'ip route add' because to there are
two different routes for 169.254.169.254:
$ ip -4 route add 192.168.1.1/32 via 0.0.0.0 dev eth0
$ ip -4 route add 192.168.1.1/32 via 10.112.140.248 dev eth0
But NetworkManager can handle such scenario successfully as it uses "ip route append".
So change cloud-init to also use "ip route append" to fix the issue:
$ ip -4 route append 192.168.1.1/32 via 0.0.0.0 dev eth0
$ ip -4 route append 192.168.1.1/32 via 10.112.140.248 dev eth0
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
RHBZ: #2003231
|
|
|
|
This attempts to standardize unit test file location under test/unittests/
such that any source file located at cloudinit/path/to/file.py may have a
corresponding unit test file at test/unittests/path/to/test_file.py.
Noteworthy Comments:
====================
Four different duplicate test files existed:
test_{gpg,util,cc_mounts,cc_resolv_conf}.py
Each of these duplicate file pairs has been merged together. This is a
break in git history for these files.
The test suite appears to have a dependency on test order. Changing test
order causes some tests to fail. This should be rectified, but for now
some tests have been modified in
tests/unittests/config/test_set_passwords.py.
A helper class name starts with "Test" which causes pytest to try
executing it as a test case, which then throws warnings "due to Class
having __init__()". Silence by changing the name of the class.
# helpers.py is imported in many test files, import paths change
cloudinit/tests/helpers.py -> tests/unittests/helpers.py
# Move directories:
cloudinit/distros/tests -> tests/unittests/distros
cloudinit/cmd/devel/tests -> tests/unittests/cmd/devel
cloudinit/cmd/tests -> tests/unittests/cmd/
cloudinit/sources/helpers/tests -> tests/unittests/sources/helpers
cloudinit/sources/tests -> tests/unittests/sources
cloudinit/net/tests -> tests/unittests/net
cloudinit/config/tests -> tests/unittests/config
cloudinit/analyze/tests/ -> tests/unittests/analyze/
# Standardize tests already in tests/unittests/
test_datasource -> sources
test_distros -> distros
test_vmware -> sources/vmware
test_handler -> config # this contains cloudconfig module tests
test_runs -> runs
|
|
Given that there are additional network management tools that we haven't
yet supported with activators, we should log a warning and continue
without network activation here, especially since this was a no-op for
years.
LP: #1948681
|
|
(#1123)
Allow #cloud-config and cloud-init query to use underscore-delimited
"jinja-safe" key aliases for any instance-data.json keys
containing jinja operator characters.
This provides a means to use Jinja's dot-notation instead of square brackets
and quoting to reference "unsafe" obtain attribute names.
Support for these aliased keys is available to both #cloud-config user-data and
`cloud-init query`.
For example #cloud-config alias access can look like:
{{ ds.config.user_network_config }}
- instead of -
{{ ds.config["user.network-config"] }}
|
|
|
|
GCE currently fetches metadata after network has come up. There's no
reason we can't fetch at init-local time, so update GCE to fetch at
init-local time to be more performant and consistent with other
datasources.
|
|
Vultr uses 169.254.169.254 for the metadata server. Some distros are
having trouble with this on IPv6 only servers because the route is
not being assigned to the link-local interface by default as it is in
other distros. This change sets that route before attempting to fetch
the metadata avoiding the current issue.
|
|
When cloud-init is configured to show SSH user key fingerprints during
boot two of the same message appears for each user. This appears to be as
the util.multi_log call defaults to send to both console directly and to
stderr (which also goes to console).
This change sends them only to console directly.
|
|
Some references were missed in the removal of the agent command
in PR #799. This simply removes the remaining references.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
"HACKING" was renamed to "CONTRIBUTING", update the PR template
URL accordingly.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
This is more consistent with other github repositories, and will prompt
a first-time contributor to read the contributing guidelines before
submitting the pull request.
Additionally, added a summary section to the top, updated some outdated
language, and removed some outdated typing guidance.
|
|
testing: monkeypatch system_info call in unit tests
system_info can make calls that read or write from the filesystem, which
should require special mocking. It is also decorated with 'lru_cache',
which means test authors often don't realize they need to be mocking.
Also, we don't actually want the results from the user's local
machine, so monkeypatching it across all tests should be reasonable.
Additionally, moved some of 'system_info` into a helper function to
reduce the surface area of the monkeypatch, added tests for the new
function (and fixed a bug as a result), and removed related mocks that
should be no longer needed.
|
|
Some Vultr Datacenters can experience latency in the connection due
to the location of one of the dependant api's. The timouts need to be
adjusted so this isn't a failure in the future.
|
|
LXD now adds cloud-init scoped configuration keys network-config,
user-data and vendor-data. The existing user.user-data,
user.vendor-data, user.network-config and meta-data will be
deprecated in newer LXD.
cloud-init will prefer LXD config keys cloud-init.* keys above
user.* keys even if both are present. Warnings will be emitted
for ignored user.* keys if cloud-init.* overrides are present.
Expectation is that the configuration user.network-config,
user.meta-data, user.user-data and user.vendor-data* keys should
not be present at the same time as the comparable cloud-init.* keys.
|
|
For Debian, the network configure file was named
/etc/network/interfaces.d/50-cloud-init, not the 50-cloud-init.cfg,
related to
https://github.com/canonical/cloud-init/blob/62721ae71057530e41779ff02ce578b7b802a60f/cloudinit/distros/debian.py#L56
the static IP customization on Debian will fail owing to
"source /etc/network/interfaces.d/*.cfg".
This change will fix this issue.
LP: #1950136
|
|
|
|
Also simplify a path and fix a spelling error while in the file
|
|
Add growpart integration test and associated unit tests
Additionally, a small runcmd check for a commented line.
|
|
Move more tests into test_combined.py and remove the CI mark from module
tests that aren't updated often or don't represent core functionality.
|
|
- Added to list of expected warnings on Oracle when opc user has
no ssh key
- Added retries to tests that read from syslog as that can sometimes
take time to reflect in the log
- Updated test_apt.py to remove proxy info into its own test as that
can cause failures in updating, which will immediately traceback
out of the module and prevent us from running further class tests
- Updated test_apt.py to use a more updated ppa in the test_keyserver
- Added basic rsyslog test to test_combined.py
- Added basic puppet test as test_puppet.py
|
|
On Bionic and Xenial, pycloudlib sets user.vendor-data config in lxd
to ensure that lxd-agent is setup on those images.
Adapt the lxd_discovery integration test to assert the appropriate
user.vendor-data config key exists if we are on xenial or bionic.
Also add assertions that /var/lib/cloud/nocloud-net/meta-data still
exists in the images because we want NoCloud to be a viable fallback
datasource if LXD config security.lxddev = false or LXD datasource
discovery encountered an unexpected error.
|
|
Integration test runs get unique log directories at
/tmp/cloud_init_test_logs/$DATE_TIME. Make
/tmp/cloud_init_test_logs/last always point to the most recent
integration test directory.
|
|
This patch address an issue where the use of the "set-name"
directive caused the networkd renderer to fail.
LP: #1949407
|
|
Currently any attempt to run an apt command while another process holds
an apt lock will fail. We should instead wait to acquire the apt lock.
LP: #1944611
|
|
Chef tests attempt to reach out to test URLs, which will get blocked by
our on our openstack installs.
|
|
Whenever "apk upgrade" is triggered also use the "--available" and
"--update-cache" options to ensure that an up-to-date packages list
is used.
|
|
Bump the version in cloudinit/version.py to 21.4 and
update ChangeLog.
LP: #1949405
|
|
During reprovisioning, VM network will change. fallback nic
should be cleared after use so that it can be re-evaluated after
reprovisioning
|
|
Without UDF support, DS Azure cannot mount the provisioning ISO,
which contains platform metadata necessary to support
pre-provisioning. The required metadata is made available in IMDS
starting with api version 2021-08-01. This change will leverage IMDS
to obtain the required metadata to support pre-preprovisioning if
provisioning ISO was not available.
|
|
In our integration tests, a few tests were modifying the environment and
then calling 'install_new_cloud_init'. This is problematic because it
updates the environment for all future tests.
Other instances of 'install_new_cloud_init' aren't problematic because
they aren't modifying the underlying environment.
|
|
Add DataSourceLXD which knows how to talk to the dev-lxd socket to
obtain all instance metadata API:
https://linuxcontainers.org/lxd/docs/master/dev-lxd.
This first branch is to deliver feature parity with the existing
NoCloud datasource which is currently used to intialize LXC instances
on first boot.
Introduce a SocketConnectionPool and LXDSocketAdapter to support
performing HTTP GETs on the following routes which are surfaced by the
LXD host to all containers:
http://unix.socket/1.0/meta-data
http://unix.socket/1.0/config/user.user-data
http://unix.socket/1.0/config/user.network-config
http://unix.socket/1.0/config/user.vendor-data
These 4 routes minimally replace the static content provided in the
following nocloud-net seed files:
/var/lib/cloud/nocloud-net/{meta-data,vendor-data,user-data,network-config}
The intent of this commit is to set a foundation for LXD socket
communication that will allow us to build network hot-plug features
by eventually consuming LXD's websocket upgrade route 1.0/events to
react to network, meta-data and user-data config changes over time.
In the event that no custom network-config is provided, default to the
same network-config definition provided by LXD to the NoCloud
network-config seed file.
Supplemental features above NoCloud datasource:
surface all custom instance data config keys via cloud-init query ds
which aids in discoverability of features/tags/labels as well as
conditional #cloud-config jinja templates operations based on custom
config options.
TBD: better cloud-init query support for dot-delimited keys
|
|
Don't throw an exception when mirror arch is unspecified.
|
|
When we added the install hotplug module, we forgot to update the
redhet/cloud-init.spec.in file and allow for execution on /usr/libexec.
This PR adds that functionality.
|
|
|
|
Also, add the "signed by" option to source definitions. This enables
users to limit the scope of trust for individual keys.
LP: #1836336
|
|
This was fixed in 1bbc4908ff7a2be19483811b3b6fee6ebc916235
|
|
When ssh host keys are generated during initial boot the full output of
ssh-keygen, including the randomart for the key, is displayed on the
console for each of the generated key types, which takes up a large
amount of screen output (17 lines per key type).
With this change ssh-keygen output is still displayed by default.
Setting ssh_quiet_keygen to True will prevent ssh-keygen output from
appearing. If only the fingerprints of the host keys should be
displayed then this can be achieved using the existing
emit_keys_to_console and/or ssh_fp_console_blacklist settings.
|
|
This commit removes automatically installing udev rules for hotplug
and adds a module to install them instead.
Automatically including the udev rules and checking if hotplug was
enabled consumed too many resources in certain circumstances. Moving the
rules to a module ensures we don't spend extra extra cycles on hotplug
if hotplug functionality isn't desired.
LP: #1946003
|
|
The Alpine /etc/hosts template results in a file where the long form of
names (including localhost) come before the short form. This means that
when running tools like 'netstat' and 'ss' which convert IP address to
names that their output will show 'localhost.localdomain' rather than
'localhost.' This patch swaps the order of the short and long form names
so such utils will show the short form name.
It also removes several unnecessary IPv6-specific entries.
|
|
Also update travis to use python version for docs that readthedocs uses
|
|
Change DMI warning to a debug message to prevent it appearing on
console during boot of machines, such as Raspberry Pi, that do
not support DMI.
|
|
disable-sshd-keygen-if-cloud-init-active.conf (#1075)
Running 'systemd-analyze verify cloud-init-local.service'
triggers the following warning:
disable-sshhd-keygen-if-cloud-init-active.conf:8: Missing '=', ignoring line.
The string "EOF" is probably a typo, so remove it.
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
|
|
The main idea is to introduce a second module that takes care of
writing files, but in the 'final' stage.
While the introduction of a second module would allow for choosing
the appropriate place withing the order of modules (and stages),
there is no addition top-level directive being added to the cloud
configuration schema. Instead, 'write-files' schema is being extended
to include a 'defer' attribute used only by the 'write-deffered-files'
modules.
The new module 'write-deferred-files' reuses as much as
possible of the 'write-files' functionality.
|
|
|
|
In jsonschema 4, hostname validation was changed to have an optional
dependency on the fqdn package. Since we don't have this dependency
in cloud-init, attempting this validation will no longer fail for
a string that isn't a valid hostname.
|
|
In some of the cases, the system-product-name is just google.
This is useful incase of nocloud where we use the disk to load the datasource
|
