| Age | Commit message (Collapse) | Author |
|---|
|
(SC-191) (#955)
This should enable us to remove the cloud-tests entirely.
|
|
Implement missing device_aliases feature
The device_aliases key has been documented as part of disk_setup for
years, however the feature was never implemented. This implements the
feature as documented allowing usercfg (rather than dsconfig) to create
a mapping of device names.
This is not to be confused with disk_aliases, a very similar map but
existing solely for use by datasources.
LP: #1867532
|
|
test_ssh_import_id.py occassionally fails because cloud-init finishes
before the keys have been fully imported. A retry has been added to the
test.
|
|
Adds a udev script which will invoke a hotplug hook script on all net
add events. The script will write some udev arguments to a systemd FIFO
socket (to ensure we have only instance of cloud-init running at a
time), which is then read by a new service that calls a new 'cloud-init
devel hotplug-hook' command to handle the new event.
This hotplug-hook command will:
- Fetch the pickled datsource
- Verify that the hotplug event is supported/enabled
- Update the metadata for the datasource
- Ensure the hotplugged device exists within the datasource
- Apply the config change on the datasource metadata
- Bring up the new interface (or apply global network configuration)
- Save the updated metadata back to the pickle cache
Also scattered in some unrelated typing where helpful
|
|
Python 3.6 added a new `policy` attribute to `MIMEMultipart`.
MIMEMultipart may be part of the cached object pickle of a datasource.
Upgrading from an old version of python to 3.6+ will cause the
datasource to be invalid after pickle load.
This commit uses the upgrade framework to attempt to access the mime
message and fail early (thus discarding the cache) if we cannot.
Commit 78e89b03 should fix this issue more generally.
|
|
defined in AuthorizedKeysFile (#937)
This patch aims to fix LP1911680, by analyzing the files provided
in sshd_config and merge all keys into an user-specific file. Also
introduces additional tests to cover this specific case.
The file is picked by analyzing the path given in AuthorizedKeysFile.
If it points inside the current user folder (path is /home/user/*), it
means it is an user-specific file, so we can copy all user-keys there.
If it contains a %u or %h, it means that there will be a specific
authorized_keys file for each user, so we can copy all user-keys there.
If no path points to an user-specific file, for example when only
/etc/ssh/authorized_keys is given, default to ~/.ssh/authorized_keys.
Note that if there are more than a single user-specific file, the last
one will be picked.
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Co-authored-by: James Falcon <therealfalcon@gmail.com>
LP: #1911680
RHBZ:1862967
|
|
summary: Clear cache when a Python version change is detected
When a distribution gets updated it is possible that the Python version
changes. Python makes no guarantee that pickle is consistent across
versions as such we need to purge the cache and start over.
Co-authored-by: James Falcon <therealfalcon@gmail.com>
|
|
Control is currently limited to boot events, though this should
allow us to more easily incorporate HOTPLUG support. Disabling
'instance-first-boot' is not supported as we apply networking config
too early in boot to have processed userdata (along with the fact
that this would be a pretty big foot-gun).
The concept of update events on datasource has been split into
supported update events and default update events. Defaults will be
used if there is no user-defined update events, but user-defined
events won't be supplied if they aren't supported.
When applying the networking config, we now check to see if the event
is supported by the datasource as well as if it is enabled.
Configuration looks like:
updates:
network:
when: ['boot']
|
|
the above option allows the user to control the behavior of a distro
hostname selection if both short hostname and FQDN are supplied.
If `prefer_fqdn_over_hostname` is true the FQDN will be selected as
hostname; if false the hostname will be selected
LP: #1921004
|
|
The current method of running a background sleep until travis is
finished is causing integration test runs to pass even when they should
be failing.
Instead, update the code to emit dots itself.
|
|
When output of SSH host keys and/or SSH fingerprints are disabled for
all keys do not display headers and footers.
Prevent risk of message text being interpreted as "logger" option by
appending "--" to logger options.
Correct syslog output that was tagged with "ec2" regardless of DataSource
in use. Now use "cloud-init" tag instead.
Various "shellcheck" corrections.
Add testcase for disabled output of SSH host keys.
|
|
Prior to this commit, when a user specified configuration which would
generate random passwords for users, cloud-init would cause those
passwords to be written to the serial console by emitting them on
stderr. In the default configuration, any stdout or stderr emitted by
cloud-init is also written to `/var/log/cloud-init-output.log`. This
file is world-readable, meaning that those randomly-generated passwords
were available to be read by any user with access to the system. This
presents an obvious security issue.
This commit responds to this issue in two ways:
* We address the direct issue by moving from writing the passwords to
sys.stderr to writing them directly to /dev/console (via
util.multi_log); this means that the passwords will never end up in
cloud-init-output.log
* To avoid future issues like this, we also modify the logging code so
that any files created in a log sink subprocess will only be
owner/group readable and, if it exists, will be owned by the adm
group. This results in `/var/log/cloud-init-output.log` no longer
being world-readable, meaning that if there are other parts of the
codebase that are emitting sensitive data intended for the serial
console, that data is no longer available to all users of the system.
LP: #1918303
|
|
The apt default test wasn't ported over from cloud-tests correctly.
uri should be specified in the test, but it was not, so the test
failed on openstack (and likely other platforms) because without
a specified uri, the default uri will vary by platform. I separated
this uri test out into a separate test function.
Also add openstack specific test for apt configuration with no uri.
Other platform-specific tests should be added here over time.
|
|
Newer verisons of /etc/sudoers prefer @includedir over
#includedir. Ensure we handle that properly and don't include an
additional #includedir when one isn't warranted.
|
|
* Xenial issue
The `apt-key finger` format changed since Xenial. Sample Xenial output:
pub 4096R/991BC93C 2018-09-17
Key fingerprint = F6EC B376 2474 EDA9 D21B 7022 8719 20D1 991B
Sample Focal output:
pub rsa4096 2016-04-12 [SC]
EB4C 1BFD 4F04 2F6D DDCC EC91 7721 F63B D38B 4796
What didn't change is the format of the key fingerprint, which should be
enough to ensure that the right key is in place across all the supported
releases.
* Hirsute issue
TestApt::test_ppa_source also fails on Hirsute because of a difference
in how the PPA keys are added. On Focla this command:
add-apt-repository ppa:simplestreams-dev/trunk
install /etc/apt/trusted.gpg.d/simplestreams-dev_ubuntu_trunk.gpg, while
on Hirsute the file is names simplestreams-dev-ubuntu-trunk.gpg. The
filename is part of the `apt-key finger` output, and this the test
fails. Only checking for the presence of the key fingerprint in apt-key
also covers this case.
LP: #1916629
|
|
Specifically:
ssh:
emit_keys_to_console: false
We also port the cc_keys_to_console cloud tests to the new integration
testing framework, and add a test for this new option.
LP: #1915460
|
|
|
|
`test_seed_random_data.py` was failing on openstack as openstack
provides additional binary seed data to the end of the specified file.
The test has been changed to only read the ascii porition of
seed file.
|
|
|
|
pycloudlib no longer raises exceptions when cloud-init fails to start,
and the API has been updated accordingly. Changes have been made to
integration tests accordingly
|
|
The documentation did not mention that the given data may not be the
exact string written: the cloud's random data may be added to it.
Additionally, the documentation of the command key was incorrect.
test_seed_random_data was updated to check that the given data is a
prefix of the written data, to match cloud-init's expected (and, now,
documented) behaviour.
LP: #1911227
|
|
|
|
On xenial, the bridge test fails because xenial's LXD doesn't include
the `network` subcommand. On bionic, the bridge test fails within
containers, because LXD isn't able to manipulate the host kernel as it
expects.
(focal and later do run successfully in containers, but we don't have a
good way of expressing that presently.)
|
|
This currently covers functionality added in #575
|
|
|
|
Ensure gpg is called with --no-tty flag.
Also, refactored the "ordered_items_in_text" to assert if the line
is missing and provide a more useful error message.
|
|
Also introduce the `unstable` mark, to allow us to land tests which
run inconsistently (such as this one).
|
|
This introduces an optional, more complex OS_IMAGE format (`<image
id>::<os>::<release>`) which allows the specification of the OS/OS
release which the given image ID corresponds to. This information is
used to skip tests which do not apply to the image.
This commit is comprised of the following discrete changes:
* introduce the IntegrationImage class, to handle parsing and storing
the new OS_IMAGE format
* support inferring the OS and OS release of Ubuntu series, so that we
can continue to set OS_IMAGE to just a series name and have test
skipping work
* add documentation on Image Selection to integration_tests.rst
* introduce the actual skipping behaviour based on OS marks
* apply the `ubuntu` mark to all tests that should be skipped on
non-Ubuntu operating systems
|
|
|
|
Tests previously assumed that when executing commands and transferring
files that user will have root permissions. This change updated
integration testing infrastructure so that is true.
|
|
Existing config writes keys to /etc/ssh after deleting files matching
a glob that includes certificate files. Since sshd looks for
certificates in the same directory as the keys, a host certificate
must be placed in this directory. This update enables the certificate's
contents to be specified along with the keys.
Co-authored-by: jonathan lung <lungj@heresjono.com>
Co-authored-by: jonathan lung <jlung@kepler.space>
|
|
This introduces the "ci" mark, used to indicate a test which should run
as part of our CI integration testing run and the integration-tests-ci
tox environment, which runs only those tests. Travis has been adjusted
to use this tox environment.
(All current module tests have been marked with the "ci" mark, but the
one bug test that we have has not.)
|
|
Translate the following tests from `cloud_tests` to the new integration test framework:
* test_runcmd.py
* seed_random_data.py
* set_hostname.py
* set_hostname_fqdn.py
* snap.py
* ssh_auth_key_fingerprints_disable.py
* ssh_auth_key_fingerprints_enable.py
* ssh_import_id.py
* ssh_keys_generate.py
* ssh_keys_provided.py
* timezone.py
* write_files.py
|
|
Specifically:
* `apt_configure_sources_list`
* `ntp_servers`
* `set_password_list`
* `users_groups`
Although not currently run in Travis, `set_password_list_string` was
ported over alongside `set_password_list` (as `test_set_password`).
|
