| Age | Commit message (Collapse) | Author |
|---|
|
This moves the base test case classes into into cloudinit/tests and
updates all the corresponding imports.
|
|
This branch is a prerequisite for IPv6 support in AWS by allowing Ec2
datasource to query the metadata source version 2016-09-02 about whether
or not it needs to configure IPv6 on interfaces. If version 2016-09-02
is not present, fallback to the min_metadata_version of 2009-04-04. The
DataSourceEc2Local not run on FreeBSD because dhclient in doesn't
support the -sf flag allowing us to run dhclient without filesystem
side-effects.
To query AWS' metadata address @ 169.254.169.254, the instance must have
a dhcp-allocated address configured. Configuring IPv4 link-local
addresses result in timeouts from the metadata service. We introduced a
DataSourceEc2Local subclass which will perform a sandboxed dhclient
discovery which obtains an authorized IP address on eth0 and crawl
metadata about full instance network configuration.
Since ec2 IPv6 metadata is not sufficient in itself to tell us all the
ipv6 knownledge we need, it only be used as a boolean to tell us which
nics need IPv6. Cloud-init will then configure desired interfaces to
DHCPv6 versus DHCPv4.
Performance side note: Shifting the dhcp work into init-local for Ec2
actually gets us 1 second faster deployments by skipping init-network
phase of alternate datasource checks because Ec2Local is configured in
an ealier boot stage. In 3 test runs prior to this change: cloud-init
runs were 5.5 seconds, with the change we now average 4.6 seconds.
This efficiency could be even further improved if we avoiding dhcp
discovery in order to talk to the metadata service from an AWS
authorized dhcp address if there were some way to advertize the dhcp
configuration via DMI/SMBIOS or system environment variables.
Inspecting time costs of the dhclient setup/teardown in 3 live runs the
time cost for the dhcp setup round trip on AWS is:
test 1: 76 milliseconds
dhcp discovery + metadata: 0.347 seconds
metadata alone: 0.271 seconds
test 2: 88 milliseconds
dhcp discovery + metadata: 0.388 seconds
metadata alone: 0.300 seconds
test 3: 75 milliseconds
dhcp discovery + metadata: 0.366 seconds
metadata alone: 0.291 seconds
LP: #1709772
|
|
If the system configure default in /etc/default/locale is set to the same
value that is provided for cloud-init's "locale" setting, then do not
re-run locale-gen. This allows images built with a locale already
generated to not re-run locale-gen (which can be very heavy).
Also here is a fix to invoke update-locale correctly and remove the
internal writing of /etc/default/locale. We were calling
update-locale <locale>
This ends up having no affect. The more correct invocation is:
update-locale LANG=<locale>
Also added some support here should we ever want to change setting
LANG to setting LC_ALL (or any other key).
Lastly, a test change to allow us to use assert_not_called from mock.
Versions of mock in CentOS 6 do not have assert_not_called.
|
|
This includes a few fixes found when testing with python 3.6.
- fix eni renderer when target is None
This just uses the util.target_path() in the event that target is None.
- change test cases to not rely on the cached result of
util.get_cmdline() and other cached globals. Update the base TestCase
to unset that cache.
- mock calls to system_is_snappy from the create_users test cases.
- drop unused _pp_root in test_simple_run.py
LP: #1703697
|
|
Unix file modes are usually represented as octal, but they were being
interpreted as decimal, for example 0o644 would be printed as '420'.
Reviewed-by: Tom Kirchner <tjk@amazon.com>
|
|
cloud-config files are very flexible and permissive.
This adds a jsonsschema definition to the cc_ntp module and validation
functions in cloudinit/config/schema which will log warnings about
invalid configuration values in the ntp section.
A cmdline tools/cloudconfig-schema is added which can be used in our dev
environments to quickly attempt to exercise the ntp schema.
It is also exposed as a main in cloudinit.config.schema.
(python3 -m cloudinit.config.schema)
LP: #1692916
|
|
Any CiTestCase subclass can now set a class attribute with_logs = True and
tests can now make assertions on self.logs.getvalue(). This branch
restructures a bit of cc_ntp module to get better test coverage of the
module. It also restructures the handler_cc_ntp unit tests to avoid nested
mocks where possible. Deeply nested mocks cause a couple of issues:
- greater risk: mocks are permanent within the scope, so multiple
call-sites could be affected by package mocks
- less legible tests: each mock doesn't advertise the actual call-site
- tight coupling: the unit test logic to tightly bound to the actual
implementation in remote (unrelated) modules which makes it more
costly to maintain code
- false success: we should be testing the expected behavior not specific
remote method names as we want to know if that underlying behavior
changes and breaks us.
LP: #1692794
|
|
Recent core snap images (edge channel revision 1886) do not contain the
previously known files used to detect that a system is ubuntu core.
The changes here are to look in 2 additional locations to determine
if a system is snappy.
LP: #1689944
|
|
This adds several unit tests for ds-identify, and fixes a bug
in Ec2 detection that I found while writing these tests.
The method of testing is to use the ds-identify code as a shell
library. The TestDsIdentify:call basically does:
* populate a (temp) directory with files that represent what
ds-identify would see in /sys or other locations it reads.
* create a file '_shwrap' that replaces the 3 programs that are executed
in ds-identify code path. It supports setting their stdout, stderr,
and exit code.
* set the default policies explicitly (DI_DEFAULT_POLICY) so we can
support testing different builtins. This is necessary because the
Ubuntu branches patch the builtin value. If we did not explicilty set
it, then testing there would fail.
* execute sh to source the script and call its main.
|
|
The test_user_data_normalize and test_net files had gotten
the executable bit set, and thus are skipped by nose by default.
We could set run with the --exe flag, but they should not have
gotten this way.
Other changes here:
* replace TempDirTestCase with CiTestCase, which has some nice
tmp_dir() and tmp_path() functions. Going forward the intent
is to have CiTestCase be the base test case for tests.
* test_net: switch to CiTestCase and fix usage that was
silently broken, because of exe bit.
* populate_dir: return the list of files that it writes rather
than having no return value.
* CiTestCase:
* support tmp_path("foo") that returns a full path to 'foo'
under a tmpdir.
* add tmp_dir() to get a temp dir and clean up.
|
|
No shipping ubuntu has a python 3.4 that is less than 3.4.2.
Remove this workaround to avoid unnecessary complexity.
This reverts 04a60cf949.
|
|
Failures to load the kernel command line's url (cloud-config-url=)
would previously get swallowed. This should make it much more
obvious when that happens. With logging going to expected places
at sane levels (WARN will go to stderr by default).
|
|
This has been a recurring ask and we had initially just made the change to
the cloud-init 2.0 codebase. As the current thinking is we'll just
continue to enhance the current codebase, its desirable to relicense to
match what we'd intended as part of the 2.0 plan here.
- put a brief description of license in LICENSE file
- put full license versions in LICENSE-GPLv3 and LICENSE-Apache2.0
- simplify the per-file header to reference LICENSE
- tox: ignore H102 (Apache License Header check)
Add license header to files that ship.
Reformat headers, make sure everything has vi: at end of file.
Non-shipping files do not need the copyright header,
but at the moment tests/ have it.
|
|
The correct order of precedence when reading the base config:
builtin config
system config
kernel command line provided config.
This reverts commit 63501f44, which actually broke the behavior it
reported to fix. It also adds some unit tests to ensure this behavior
is not broken again.
LP: #1582323
|
|
Many of the unit tests in test_data would inadvertantly read the
system's /etc/cloud/cloud.cfg and /etc/cloud/cloud.cfg.d.
This was first noticed on a system deployed by MAAS, where
files in /etc/cloud/cloud.cfg.d/ are root read-only.
This changes those tests to actually make use of
FilesystemMockingTestCase functionality and adds 'reRoot()' to that
class which is easier to use for at least this use case.
LP: #1635350
|
|
Change atomic_helper.write_file to have same same signature as write_file.
Add some simple unit tests for atomic_helper.
|
|
Add vendor-data support to maas which will behave like the openstack
vendor-data does. Data returned from maas must be yaml loadable.
Also update the main in DataSourceMAAS to "just work" on a maas
deployed system.
LP: #1612313
|
|
test runs to the point where it did, think I got most of the changes
incorporated.
|
|
|
|
|
|
|
|
|
|
If the datasource's instance id contained a '/' then the instance_id path
would not be as expected under /var/lib/cloud/instances/instance_id.
LP: #1575938
|
|
Patch FilesystemMockingTestcase.patchOS to support methods taking more
than a single path argument. This is required in order to properly mock
`os.symlink`, which takes two path arguments.
|
|
HTTPretty can causes hangs on Python 3.4.2 (and maybe Python 3.4.1), due
to a Python bug (fixed in Python 3.4.3). This works around the problem
in the appropriate Python versions.
See https://github.com/gabrielfalcao/HTTPretty/pull/193 and
https://github.com/gabrielfalcao/HTTPretty/issues/221 for details.
|
|
UrlResponse: biggest change... make readurl return bytes, making user
know what to do with it.
util: add load_tfile_or_url for loading text file or url
as read_file_or_url now returns bytes
ec2_utils: all meta-data is text, remove non-obvious string translations
DigitalOcean: adjust for ec2_utils
DataSourceGCE, DataSourceMAAS: user-data is binary other fields are text.
openstack.py: read paths without decoding to text. This is ok as paths
other than user-data are json, and load_json will handle
load_file still returns text, and that is what most things use.
|
|
This should fix the Azure data source on Python 3, and is appropriate as
XML shouldn't really be read as a string.
|
|
|
|
painful, and not perfect, but at this point the output builds
on a vivid system python2 (bddeb --python2) or python3.
* remove use of cheetah by bddeb in favor of builtin renderer
* add '--python2' flag to bddeb and knowledge of python 2 and python3
package names.
* read-dependencies can now read test-requirements also.
* differenciate from build-requirements and runtime requirements.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The digital ocean datasource test is using assertIs which
is only created/existent on py2.7, so for the older py2.6
we need to add similar logic so that the test works correctly
there.
|
|
previously this would fail:
http_proxy=http://foo.bar make test
now it will pass. This works around a bug where httpretty is not able to
patch http operations if http_proxy is set.
https://github.com/gabrielfalcao/HTTPretty/issues/122
|
|
A few of the current tests have been continually failing
in python 2.6 based systems, due to lack of unit test
functions that are now added to ensure we can run the
unit tests (and not have to ignore those failures) on
python 2.6
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This function can be used to ensure that mocker objects
are restored and verified during usage if exceptions are
thrown while the mock object is being used.
Ensure it is used in the config drive test when multiple
mock objects are being created and restored.
LP: #1098430
|
|
- Add a helper testcase class that can add additional features into the
unit test class as we need for features that are useful to have which
starts with features that are missing including assertIn and assertNotIn
LP: #1078473
|
|
|
