| Age | Commit message (Collapse) | Author |
|---|
|
In CI run against pylint 2.9.3 and fix occurrences of:
- W0237 (arguments-renamed)
- W0402 (deprecated-module)
The W0402 deprecated-module was about module `imp`:
cloudinit/patcher.py:9: [W0402(deprecated-module), ]
Uses of a deprecated module 'imp'
The imp module is deprecated and replaced by importlib, which according
to the documentation has no replacement for acquire_lock() and
release_lock(), which are the only reason why `imp` is imported.
Nothing about the code using this lock that actually requires it.
Let's remove the locking code and the import altogether.
Dropping the locking makes patcher.patch() an empty wrapper around
_patch_logging(). Rename _patch_logging() to patch_logging() and
call it directly instead. Drop patch().
|
|
Also new jenkins tox definition
|
|
In #856 we added the ability to use partprobe instead of blockdev for
reading partitions. Test that partprobe succeeds where blockdev fails.
Also add a mechanism to our integration tests to allow a callable to be
called between `lxc init` and `lxc start`
|
|
Control is currently limited to boot events, though this should
allow us to more easily incorporate HOTPLUG support. Disabling
'instance-first-boot' is not supported as we apply networking config
too early in boot to have processed userdata (along with the fact
that this would be a pretty big foot-gun).
The concept of update events on datasource has been split into
supported update events and default update events. Defaults will be
used if there is no user-defined update events, but user-defined
events won't be supplied if they aren't supported.
When applying the networking config, we now check to see if the event
is supported by the datasource as well as if it is enabled.
Configuration looks like:
updates:
network:
when: ['boot']
|
|
The current method of running a background sleep until travis is
finished is causing integration test runs to pass even when they should
be failing.
Instead, update the code to emit dots itself.
|
|
The apt default test wasn't ported over from cloud-tests correctly.
uri should be specified in the test, but it was not, so the test
failed on openstack (and likely other platforms) because without
a specified uri, the default uri will vary by platform. I separated
this uri test out into a separate test function.
Also add openstack specific test for apt configuration with no uri.
Other platform-specific tests should be added here over time.
|
|
This allows source'd OpenStack credentials to be used for tox tests.
|
|
pycloudlib has modified the way LXD executes tests
(https://github.com/canonical/pycloudlib/pull/114): it will always use
SSH to access them by default, instead of using `lxc exec`. This
behaviour is transparent for them majority of cloud-init's integration
tests, but some currently depend on using `lxc exec` to access instances
with (intentionally) broken networking: obviously these are not
accessible via SSH.
pycloudlib retains support for switching an instance to use `lxc exec`.
This commit introduces the `lxd_use_exec` mark, which tests can use to
indicate to the integration testing framework that they should be so
switched, and applies it to all applicable tests.
|
|
Kernel's newer than 4.15 present /sys/dmi/id/product_uuid as a
lowercase value. Previously UUID was uppercase.
Azure datasource reads the product_uuid directly as their platform's
instance-id. This presents a problem if a kernel is either
upgraded or downgraded across the 4.15 kernel version boundary because
the case of the UUID will change, resulting in cloud-init seeing a
"new" instance id and re-running all modules.
Re-running cc_ssh in cloud-init deletes and regenerates ssh_host keys
on a system which can cause concern on long-running instances that
somethingnefarious has happened.
Also add:
- An integration test for this for Azure Bionic Ubuntu FIPS upgrading from
a FIPS kernel with uppercase UUID to a lowercase UUID in linux-azure
- A new pytest.mark.sru_next to collect all integration tests related to our
next SRU
LP: #1835584
|
|
We need to allow the SSH_AUTH_SOCK environment variable through for
paramiko to be able to find the agent.
|
|
The tox pylint command was failing because of competing dependencies in
the multiple requirements files. Given that we plan on no longer
updating cloud_tests, we also no longer need to lint them.
|
|
|
|
|
|
integration_tests: add test for LP: #1898997
This introduces the `lxd_config_dict` mark, used to specify a
free-form configuration dict to LXD for tests which only run there; and
the `not_xenial` and `not_bionic` marks, used to skip tests on
xenial/bionic via a basic release skipping mechanism.
This also bumps the pycloudlib commit we depend upon, as
the latest commit includes the changes required for LXD network config
to work.
(The `lxd_config_dict` change further complicated `_client`, so a minor
refactoring is applied.)
|
|
Also introduce the `unstable` mark, to allow us to land tests which
run inconsistently (such as this one).
|
|
This introduces an optional, more complex OS_IMAGE format (`<image
id>::<os>::<release>`) which allows the specification of the OS/OS
release which the given image ID corresponds to. This information is
used to skip tests which do not apply to the image.
This commit is comprised of the following discrete changes:
* introduce the IntegrationImage class, to handle parsing and storing
the new OS_IMAGE format
* support inferring the OS and OS release of Ubuntu series, so that we
can continue to set OS_IMAGE to just a series name and have test
skipping work
* add documentation on Image Selection to integration_tests.rst
* introduce the actual skipping behaviour based on OS marks
* apply the `ubuntu` mark to all tests that should be skipped on
non-Ubuntu operating systems
|
|
Xenial tox 2.3.1 still suffers from
https://github.com/tox-dev/tox/issues/208.
As a result we cannot use testenv substvars if we want to support running
tox on xenial systems.
Drop the {[testenv:integration-tests]*} substitutions to avoid tracebacks
when running `tox -e xenial-dev` which has the signature:
"No support for the %s substitution type" % sub_type)
tox.ConfigError: ConfigError: No support for the posargs substitution
type
|
|
|
|
See #585
|
|
As the first test of this SRU cycle, this also introduces the
sru_2020_11 mark to allow us to easily identify the set of tests
generated for this SRU.
|
|
`-k` will select tests or marks whose names match. `-m` selects only
tests which have the specified mark, so is more appropriate.
|
|
This introduces the "ci" mark, used to indicate a test which should run
as part of our CI integration testing run and the integration-tests-ci
tox environment, which runs only those tests. Travis has been adjusted
to use this tox environment.
(All current module tests have been marked with the "ci" mark, but the
one bug test that we have has not.)
|
|
The cloud_tests and the integration_tests (via pycloudlib) have
conflicting requirements. This commit splits up their requirements
files, so we can accurately express the requirements for each.
|
|
* integration_tests: fix passing launch_kwargs to session_cloud.launch
* integration_tests: log the launch_kwargs before launching instances
* integration_tests: add support for specifying instance name for tests
Co-authored-by: Rick Harding <rharding@mitechie.com>
|
|
This means that the integration tests do not need to install
test-requirements.txt in order to successfully import `conftest.py`.
|
|
|
|
|
|
Changes:
tox: bump the pylint version to 2.6.0 in the default run
Fix pylint 2.6.0 W0707 warnings (raise-missing-from)
|
|
The version was bumped in c7248059dd2faaaadfbcef5c83e8e8ea166d6767 to
support running on Python 3.7+ systems. Now that we have separate
`xenial` and `xenial-dev` tox environments, we can restore the correct
pinning for `xenial` without breaking `xenial-dev` on developer
machines.
Also drop the `mock` dependency from `xenial-shared-deps`; its removal
was missed in 5f8f85bb38cc972d3d2c705a1ec73db3f690f323.
|
|
The /opc/v1/ metadata endpoints[0] are universally available in Oracle
Cloud Infrastructure and the OpenStack endpoints are considered
deprecated, so we can refactor the data source to use the OPC endpoints
exclusively. This simplifies the datasource code substantially, and
enables use of OPC-specific attributes in future.
[0] https://docs.cloud.oracle.com/en-us/iaas/Content/Compute/Tasks/gettingmetadata.htm
|
|
Specifically:
* disable E1102 in cloudinit/sources/helpers/openstack.py for reasons
described in a comment, and
* refactor `abs_join` to require at least one positional argument; this
matches os.path.join's signature, and that mismatch is what was
causing pylint to emit a warning
* bump to pylint 2.4.2
|
|
This is an improvement over indirect parameterisation for a few reasons:
* The test code is much easier to read, the mark names are much more
intuitive than the indirect parameterisation invocation, and there's
less boilerplate to boot
* The fixture no longer has to overload the single parameter that
fixtures can take with multiple meanings
|
|
test using it (#461)
caplog is only available in pytest itself from 3.0 onwards. In xenial, we only have pytest 2.8.7. However, in xenial we do have pytest-catchlog available (as python3-pytest-catchlog), so we use that where appropriate.
|
|
|
|
This fixes issues with closing brackets not matching the opening
bracket's line and continuation line under-idented for hanging indent.
|
|
Remove extra spaces after a ','
|
|
This puts an ignore on the imports not at the top of the file errors.
The reason for the ignore instead of fix is that the file is using imp
to grab a lock and patch logging before further imports are completed.
|
|
This removes the use of variables named ‘l’, ‘O’, or ‘I’. Generally
these are used in list comprehension to read the line of lines.
|
|
Instead of running pycodestyle and pyflakes seperately, use flake8 to
get the benefits of pyflakes and also stylistic checks as well as the
ability to configure the settings for the project.
|
|
This ensures that Travis will not kill our tests if fetching images is
taking a long time.
In implementation terms, this introduces a context manager which will
spin up a multiprocessing.Process in the background and print a dot to
stdout every 10 seconds. The process is terminated when the context
manager exits.
This also drop the use of travis_wait, which was being used to work
around this issue.
|
|
These libraries provide backports of Python 3's stdlib components to Python 2. As we only support Python 3, we can simply use the stdlib now. This pull request does the following:
* removes some unneeded compatibility code for the old spelling of `assertRaisesRegex`
* replaces invocations of the Python 2-only `assertItemsEqual` with its new name, `assertCountEqual`
* replaces all usage of `unittest2` with `unittest`
* replaces all usage of `contextlib2` with `contextlib`
* drops `unittest2` and `contextlib2` from requirements files and tox.ini
It also rewrites some `test_azure` helpers to use bare asserts. We were seeing a strange error in xenial builds of this branch which appear to be stemming from the AssertionError that pytest produces being _different_ from the standard AssertionError. This means that the modified helpers weren't behaving correctly, because they weren't catching AssertionErrors as one would expect. (I believe this is related, in some way, to https://github.com/pytest-dev/pytest/issues/645, but the only version of pytest where we're affected is so far in the past that it's not worth pursuing it any further as we have a workaround.)
|
|
|
|
See the added comment for details.
|
|
Now that we can distinguish between CI xenial dependencies and
needed-to-run-on-dev-machine xenial depedencies, we can return to
testing with the correct jsonpatch version.
|
|
pyflakes versions older than 2.1.0 are incompatible with Python 3.8
(which is the Python version in the current Ubuntu development release).
See https://github.com/PyCQA/pyflakes/issues/367 for details.
2.1.1 is the latest version ATM, so bump to that.
|
|
As the nose docs[0] themselves note, it has been in maintenance mode for the past several years. pytest is an actively developed, featureful and popular alternative that the nose docs themselves recommend. See [1] for more details about the thinking here.
(This PR also removes stale tox definitions, instead of modifying them.)
[0] https://nose.readthedocs.io/en/latest/
[1] https://lists.launchpad.net/cloud-init/msg00245.html
|
|
Specifically, drop it from the default list of environments that tox
will run, and from Travis.
(We retain the configuration in tox.ini for now, for any remaining
Python 2.7 needs.)
|
|
This makes it easier to debug differences in test behaviour between
Travis and local developer environments.
|
|
* docs: Add security.md to readthedocs
This enables the ability to show the security policy on both GitHub and
on the readthedocs site. To do this, enable the ability to import
Markdown based files and translate them to rst.
* Add doc-requirements.txt and update tox to use
Also removes the extra, uncessary extension addition of .md
|
|
Added Azure to cloud tests supporting upstream integration testing.
Implement the inherited platform classes, Azure configurations
to release/platform, and docs on how to run Azure CI.
|
