| Age | Commit message (Collapse) | Author |
|---|
|
Commit d3e803ad316e6796e5d83e7e8f8f4f7224b92df9 added deb-src comments to
the cloud-init apt templates. This doubled the number of matching entries
seen in /etc/apt/sources.list in apt_configure_primary integration test.
This test was really asserting that GaTech urls were present in
/etc//apt/sources.list instead of archive.ubuntu.com. Fix the test to be a
bit more flexible in case cloud-init changes its bas apt template again.
|
|
Individual skipTest or setUp SkipTest will still launch the instance.
This allows us to stop the running of the instance so we don't
waste cycles or boot systems that are known to fail.
Also replace remaining unittest usage in tests/cloud_tests/
with unittest2.
|
|
Skip lxd tests on cosmic for two reasons:
a.) bug 1795036 - 'lxd init' fails on cosmic kernel.
b.) apt install lxd installs via snap which can be slow
as that will download core snap and lxd.
|
|
Git commitish fc4b966ba928b30b1c586407e752e0b51b1031e8 changed integration
test dependencies from unittest to unittest2. Use unittest2.SkipTest in
test_chrony to avoid causing tracebacks.
|
|
Relax expectation on path to lxc and lxd. The deb path still does
install them in /usr/bin/ but that is overly pedantic.
Add a 'lxd waitready' (present since lxd 0.5) to wait until lxd
is ready before operating on it.
|
|
Commitish c7555762f3a30190ce7726b4d013bc3e83c7e4b6 changed the variable
names in instance-data.json from hyphenated to underscore delimited. In
the shuffle, meta-data -> meta_data was missed.
|
|
Cloud-init caches any cloud metadata crawled during boot in the file
/run/cloud-init/instance-data.json. Cloud-init also standardizes some of
that metadata across all clouds. The command 'cloud-init query' surfaces a
simple CLI to query or format any cached instance metadata so that scripts
or end-users do not have to write tools to crawl metadata themselves.
Since 'cloud-init query' is runnable by non-root users, redact any
sensitive data from instance-data.json and provide a root-readable
unredacted instance-data-sensitive.json. Datasources can now define a
sensitive_metadata_keys tuple which will redact any matching keys
which could contain passwords or credentials from instance-data.json.
Also add the following standardized 'v1' instance-data.json keys:
- user_data: The base64encoded user-data provided at instance launch
- vendor_data: Any vendor_data provided to the instance at launch
- underscore_delimited versions of existing hyphenated keys:
instance_id, local_hostname, availability_zone, cloud_name
|
|
Allow users to provide '## template: jinja' as the first line or their
#cloud-config or custom script user-data parts. When this header exists,
the cloud-config or script will be rendered as a jinja template.
All instance metadata keys and values present in
/run/cloud-init/instance-data.json will be available as jinja variables
for the template. This means any cloud-config module or script can
reference any standardized instance data in templates and scripts.
Additionally, any standardized instance-data.json keys scoped below a
'<v#>' key will be promoted as a top-level key for ease of reference in
templates. This means that '{{ local_hostname }}' is the same as using the
latest '{{ v#.local_hostname }}'.
Since instance-data is written to /run/cloud-init/instance-data.json, make
sure it is persisted across reboots when the cached datasource opject is
reloaded.
LP: #1791781
|
|
The snap test requires access to a proxy and currently the integration
tests do not handle this scenario. I am disabling the test until I can
loop back around and fix this.
|
|
The snap test requires access to a proxy and currently the integration
tests do not handle this scenario. I am disabling the test untill I can
loop back around and fix this.
The write_files test, specifically, the binary test is failing on cosmic
because the "binary" file we were writting was not a complete elf
executable, but we expected 'file' to identify it as such.
The change here is to simply use some 24 bytes of random, non-utf data
and check that file was written correctly via expected checksum.
|
|
The salt minion integration test as we had it did not do a whole lot
more than the unit tests on that module did. Additionally, it caused
some transient failures at least in Ubuntu 18.04.
At a future date we may choose to add an integration test that installs
salt-minion and salt server and configures it to be a better test.
LP: #1778737
|
|
In ubuntu, the salt-minion package version 2017.7.4+dfsg1-1 or later
automatically moves any seed keys from /etc/salt/pki/minion/ to
/var/lib/salt/pki/minion/. Fix integration tests to collect
either files in either /etc/salt/pki/minion/ or
/var/lib/salt/pki/minion/.
|
|
A fix for chrony support per LP: #1589780 is not expected in Artful or
older series. Skip the chrony suite of tests when running on a container
and ubuntu series represented is <= artful as errors are expected.
|
|
package_update_upgrade_install was failing as htop is now included in
Bionic images. Switch this test to install 'sl' instead.
ca_certs integration test fails on cert_count test because bionic
update-ca-certificates on bionic generates less symlinks for a given cert.
Integration tests now collect dpkg-query --show output on every instance.
Add a new assertPackageInstalled helper method which finds the package or
package version installed on the instance.
Adapt existing byobu, package_update_upgrade_install, ntp and salt_minion
tests to use assertPackageInstalled method.
LP: #1769985
|
|
This enables warnings produced by pylint for unused variables (W0612),
and fixes the existing errors.
|
|
Python has deprecated these invalid string literals now
https://bugs.python.org/issue27364
and pycodestyle is identifying them with a W605 warning.
https://github.com/PyCQA/pycodestyle/pull/676
So basically, any use of \ not followed by one of [\'"abfnrtv]
or \ooo (octal) \xhh (hex) or a newline is invalid. This is most
comomnly seen for us in regex. To solve, you either:
a.) use a raw string r'...'
b.) correctly escape the \ that was not intended to be interpreted.
|
|
Add a base NTP client configuration dictionary and allow Distro
specific changes to be merged. Add a select client function which
implements logic to preferr installed clients over clients which
need to be installed. Also allow distributions to override the
cloud-init defaults.
LP: #1749722
|
|
Fix integraiton test logic for ec2 to look for network and
availability-zone data under the key path
'ds'=>'meta-data' instead of just 'ds' when parsing instance-data.json.
|
|
LP: #1420018
|
|
Support installing and configuring snaps on ubuntu systems. Now,
cloud-config files can provide a list or dictionary of snap:assertions
which will be allow configuration of snapd on a system via 'snap ack'
calls. The snap:commands configuration option supports arbitrary system
commands intended to interact with snappy's cli. This allows users to run
arbitrary snappy commands to create users, download, install and
configure snap packages and snapd.
This branch also deprecates old snappy and snap_config modules leaving
warnings in documentation and runtime for consumers of these modules.
Deprecated snap* modules will be dropped in cloud-init v.18.2 release.
|
|
Previously the module was not working under FreeBSD due to a different
package name and some different paths. The module now has OS specific
default values which can even be customized via corresponding cloud config
variables.
LP: #1721503
|
|
While salt grains can be configured in the minion config file, it is
usually better to configure it in the /etc/salt/grains file.
This allows that to be done.
|
|
This adds collection a gzip compressed systemd journal on systemd systems.
The file can later be reviewed with:
zcat system.journal.gz > system.journal
journalctl --file=system.journal [-o short-monotonic ..]
To support this:
* modify test harness infrastructure to not assume content is utf-8.
* fix lxd platform to support make '_execute' return bytes rather
than a string. https://github.com/lxc/pylxd/issues/268
Also switched the base collectors to use /bin/sh as others already did.
|
|
Tests run on EC2 would successfully resolve the ipv4 dns address
and that caused false positives on failure reports.
Basically, dns lookup of 172.16.15.14 would return
ip-172-16-15-14.us-east-2.compute.internal.
which then shows up in the ntpq output unless you provide -n.
|
|
Due to additional platforms getting added this test was not taking into
account platform specific mirrors nor was it checking that no additional
entries were added.
|
|
The motivation for this is that
a.) 1.7.1 runs with python 3.6 (bionic)
b.) we want to run pylint on tests/ and tools for the same reasons
that we want to run it on cloudinit/
The changes are described below.
- Update tox.ini to invoke pylint v1.7.1.
- Modify .pylintrc generated-members ignore mocked object members (m_.*)
- Replace "dangerous" params defaulting to {}
- Fix up cloud_tests use of platforms
- Cast some instance objects to with dict()
- Handle python2.7 vs 3+ ConfigParser use of readfp (deprecated)
- Update use of assertEqual(<boolean>, value) to assert<Boolean>(value)
- replace depricated assertRegexp -> assertRegex
- Remove useless test-class calls to super class
- Assign class property accessors a result and use it
- Fix missing class member in CepkoResultTests
- Fix Cheetah test import
|
|
Cloud-init integration tests should not depend on a curtin test ppa.
We already had a cloud-init test ppa for explicitly this purpose.
Just use it instead.
|
|
We had used some dns records in i9n.brickies.net (my personal domain)
as a temporary solution until we got names registered in the cloud-init.io
namespace.
We now have CNAME records for:
ubuntu.i9n.cloud-init.io
cloudinit1.cloud-init.io
cloudinit2.cloud-init.io
|
|
Integration test harness changes:
* Enable collection of console log in nocloud-kvm and lxd.
* Collect the console log to results for all test runs.
* change 'tmpfile' to pick name locally instead of using 'mktemp'.
* drop the 'instance' attribute from nocloud-kvm Image and
demote LXDImage.instance to a private attribute.
This is because Images do not actually have instances.
(LXDImage internally uses a booted system to modify the image).
* Add 'TargetBase' as a superclass of Image and Instance providing
implementations of execute, read_data, write_data, pull_file,
and push_file. These all depend on an implementation of _execute.
* Improve '_execute' implementations to support accepting stdin.
* execute supports 'rcs=False' meaning 'do not raise exception'.
* Drop support for pylxd < 2.2. older versions cannot determine
exit code of 'execute', which makes them unusable.
* make NoCloudKVMInstance._execute run as root via sudo. This required
some changes so that 'hostname' could be reverse-looked up in order
to avoid sudo taking a long time (~20 seconds).
* re-use existing ssh connection in nocloud-kvm.
Test changes here:
* do not use /tmp, but rather /var/tmp (LP: #1707222)
* make keys_to_console assertions more strict.
* change user test cases to always add default (ubuntu) user
so that nocloud-kvm's execute which operates over ssh can work.
|
|
There was fallout in a full integration test run from my adding of
test_no_warnings_in_log which asserted that there could not be a WARNING
found in the /var/log/cloud-init.log
This fixes 2 of the cases:
* TestCommandOutputSimple had a valid WARNING written, so adjust its
test case to allow for that.
* TestLxdDir had a valid config in the test but the module would
log a WARNING, so fix the module.
Also updates lxd unit tests to look for WARN themselves.
|
|
Tests are currently run by creating a temporary subclass of each class
and then executing it (in get_suites). When running the tests suite
the output would contain the temporary class name. That was less than
useful, and made batch runs almost impossible to identify which
test case had an error.
This change goes from output of:
FAIL: test_no_warnings_in_log \
(tests.cloud_tests.testcases.get_suite.<locals>.tmp)
To
FAIL: test_no_warnings_in_log \
(tests.cloud_tests.testcases.modules.ntp.TestNtp)
|
|
Fix three things related to the ntp module:
1. Fix invalid cloud-config schema in the integration test which
provided empty dicts instead of emptylists for pools and servers
2. Correct logic in the ntp module to allow support for the minimal
cloud-config 'ntp:' without raising a RuntimeError. Docs and schema
definitions already describe that cloud-config's ntp can be empty.
An ntp configuration with neither pools nor servers will be
configured with a default set of ntp pools. As such, the ntp module
now officially allows the following ntp cloud-configs:
- ntp:
- ntp: {}
- ntp:
servers: []
pools: []
3. Add a simple unit test which validates all cloud-config provided to
our integration tests to ensure it adheres to any defined module
schema so as more jsonschema definitions are added, we validate our
integration test configs.
LP: #1724951
|
|
When operating in expected path, cloud-init should avoid logging with
warning. That causes 'WARNING' messages in /var/log/cloud-init.log.
By default, warnings also go to the console.
Since jsonschema is a optional dependency, and not present on xenial
and zesty, cloud-init should not warn there.
Also here:
* Add a test to integration tests to assert that there are no
warnings in /var/log/cloud-init.log.
* Update one integration test that did show warning and the related
documentation and examples.
LP: #1724354
|
|
Combine the configs and testcases directories, so all files are
together in one place. Update the test config location as well.
|
|
The locale integration test started failing with commit 0ef61b28.
This was just because the test's expectations on the content/formatting
of /etc/default/locale were too strict. The change here is to read
the file as a set of shell variables and assert that the values are
set correctly.
|
|
The test is currently importing the incorrect keyid. It specifies
the curtin developers ppa, rather than the cloud-init ppa. On
Artful this causes failures as a check is made to verify the
correct key is imported for the ppa, whereas on previous releases
only a warning was issued.
Also, change to use a full key fingerprint.
LP: #1702717
|
|
I want to be able to add additional SSH keys to my account, therefore I
should not be limiting these tests to look for one specific key. Instead
we confirm that the comment in authorized_users has the specified users.
|
|
Recent change to ntp in artful has added the sntp package whenever
ntp is installed. The tests, rather poorly, did a dpkg -l instead
of checking with `which`. This fixes the ntp tests to all use
`which` over expecting a certain number of lines using dpkg and
as a result make the tests OS independent.
|
|
Massive update to clean up and greatly enhance the integration testing
framework developed by Wesley Wiedenmeier.
- Updated tox environment to run integration test 'citest' to utilize
pylxd 2.2.3
- Add support for distro feature flags
- add framework for feature flags to release config with feature groups
and overrides allowed in any release conf override level
- add support for feature flags in platform and config handling
- during collect, skip testcases that require features not supported by
the image with a warning message
- Enable additional distros (i.e. centos, debian)
- Add 'bddeb' command to build a deb from the current working tree
cleanly in a container, so deps do not have to be installed on host
- Adds a command line option '--preserve-data' that ensures that
collected data will be left after tests run. This also allows the
directory to store collected data in during the run command to be
specified using '--data-dir'.
- Updated Read the Docs testing page and doc strings for pep 257
compliance
|
|
Snapd does not start on artful or on the versions in proposed. This
changes the behavior of the test to confirm that snapd is installed, not
that it is started, while the snap team fixes the issue (LP: ##1690880).
|
|
|
|
|
|
On systems which installed ntp and specified servers or pools in the
config ntpd didn't notice the updated configuration file and didn't
use the correct configuration. Resolve this by rendering the template
first which allows the package install to use the existing
configuration. Additionally add a service restart to handle the case
where ntp does not need to be installed but it may not have started.
Add an integration test to confirm that cc_ntp enables ntp to use the
specific servers and pools in the cloud-config.
LP: #1645644
|
|
Also add integration test. Note: this new test is not comprehensive; it
simply ensures that the example chef configuration does not blow up and
that chef seems to be installed after its completion.
This new test is disabled by default as it depends on a 3rd party
repository.
LP: #1678145
|
|
This change will add support for hashed passwords in cc_set_passwords.
It checks if a password is a hash with by checking that it matches
in fairly safe way, and also that the password does not have a ":" in it.
chpasswd needs to know if the password is hashed or not, so two lists
is created so chpasswd is feed with the correct one.
LP: #1570325
|
|
This adds an integration test for setting passwords when given
as a list rather than a string. This also updates the docs and
tests so that Random is now RANDOM as is correct.
|
|
Use inspect.getmro(mod) rather than mod.__bases__ to test if a potential
testcase class inherits from CloudTestCase. This allows testcases to be
based on CloudTestCase indirectly, adding greater flexibility to the
structure of test classes.
|
|
When testing for timezone we were testing that 'date' output would
contain 'HDT' for the current time. But after a 'spring forward', the
current time started to have 'HST'.
Instead of asking 'date' for the timezone that applies now, ask it
for a static date.
|
|
The adds in end-to-end testing of cloud-init. The framework utilizes
LXD and cloud images as a backend to test user-data passed in.
Arbitrary data is then captured from predefined commands specified
by the user. After collection, data verification is completed by
running a series of Python unit tests against the collected data.
Currently only the Ubuntu Trusty, Xenial, Yakkety, and Zesty
releases are supported. Test cases for 50% of the modules is
complete and available.
Additionally a Read the Docs file was created to guide test
writing and execution.
|
