summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2020-11-24Release 20.4 (#686)20.4James Falcon
Bump the version in cloudinit/version.py to 20.4 and update ChangeLog. LP: #1905440
2020-11-24tox: avoid tox testenv subsvars for xenial support (#684)Chad Smith
Xenial tox 2.3.1 still suffers from https://github.com/tox-dev/tox/issues/208. As a result we cannot use testenv substvars if we want to support running tox on xenial systems. Drop the {[testenv:integration-tests]*} substitutions to avoid tracebacks when running `tox -e xenial-dev` which has the signature: "No support for the %s substitution type" % sub_type) tox.ConfigError: ConfigError: No support for the posargs substitution type
2020-11-23Ensure proper root permissions in integration tests (#664)James Falcon
Tests previously assumed that when executing commands and transferring files that user will have root permissions. This change updated integration testing infrastructure so that is true.
2020-11-23LXD VM support in integration tests (#678)James Falcon
2020-11-23Integration test for fallocate falling back to dd (#681)James Falcon
See #585
2020-11-23.travis.yml: correctly integration test the built .deb (#683)Daniel Watkins
`IMAGE_SOURCE` was renamed to `CLOUD_INIT_SOURCE` before the testing framework ever landed, but I did not mirror that change in .travis.yml before it landed. This addresses that.
2020-11-23Ability to hot-attach NICs to preprovisioned VMs before reprovisioning (#613)aswinrajamannar
Adds the ability to run the Azure preprovisioned VMs as NIC-less and then hot-attach them when assigned for reprovision. The NIC on the preprovisioned VM is hot-detached as soon as it reports ready and goes into wait for one or more interfaces to be hot-attached. Once they are attached, cloud-init gets the expected number of NICs (in case there are more than one) that will be attached from IMDS and waits until all of them are attached. After all the NICs are attached, reprovision proceeds as usual.
2020-11-20Support configuring SSH host certificates. (#660)Jonathan Lung
Existing config writes keys to /etc/ssh after deleting files matching a glob that includes certificate files. Since sshd looks for certificates in the same directory as the keys, a host certificate must be placed in this directory. This update enables the certificate's contents to be specified along with the keys. Co-authored-by: jonathan lung <lungj@heresjono.com> Co-authored-by: jonathan lung <jlung@kepler.space>
2020-11-20add integration test for LP: #1900837 (#679)Daniel Watkins
As the first test of this SRU cycle, this also introduces the sru_2020_11 mark to allow us to easily identify the set of tests generated for this SRU.
2020-11-19cc_resizefs on FreeBSD: Fix _can_skip_ufs_resize (#655)Mina Galić
On FreeBSD, if a UFS has trim: (-t) or MAC multilabel: (-l) flag, resize FS fail, because the _can_skip_ufs_resize check gets tripped up by the missing options. This was reported at FreeBSD Bugzilla: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=250496 and as LP: #1901958 Rather than fixing the parser as in the patches proposed there (and attempted in #636) this pull-request rips out all of it, and simplifies the code. We now use `growfs -N` and check if that returns an error. If it returns the correct kind of error, we can skip the resize, because we either are at the correct size, or the filesystem in question is broken or not UFS. If it returns the wrong kind of error, we just re-raise it. LP: #1901958
2020-11-19DataSourceAzure: push dmesg log to KVP (#670)Anh Vo
Pushing dmesg log to KVP to help troubleshoot VM boot issues
2020-11-19Make mount in place for tests work (#667)James Falcon
IMAGE_SOURCE = 'IN_PLACE' wasn't working previously. Replaced LXD launch with an init, then mount, then start.
2020-11-19integration_tests: restore emission of settings to log (#657)Daniel Watkins
2020-11-18DataSourceAzure: update password for defuser if exists (#671)Anh Vo
cc_set_password will only update the password for the default user if cfg['password'] is set. The existing code of datasource Azure will fail to update the default user's password because it does not set that metadata. If the default user doesn't exist in the image, the current code works fine because the password is set during user create and not in cc_set_password
2020-11-18tox.ini: only select "ci" marked tests for CI runs (#677)Daniel Watkins
`-k` will select tests or marks whose names match. `-m` selects only tests which have the specified mark, so is more appropriate.
2020-11-18Azure helper: Increase Azure Endpoint HTTP retries (#619)Johnson Shi
Increase Azure Endpoint HTTP retries to handle occasional platform network blips. Introduce a common method http_with_retries in the azure.py helper, which will serve as the common HTTP request handler for all HTTP requests with the Azure endpoint. This method has builtin retries and reporting diagnostics logic.
2020-11-18DataSourceAzure: send failure signal on Azure datasource failure (#594)Johnson Shi
On systems where the Azure datasource is a viable platform for crawling metadata, cloud-init occasionally encounters fatal irrecoverable errors during the crawling of the Azure datasource. When this happens, cloud-init crashes, and Azure VM provisioning would fail. However, instead of failing immediately, the user will continue seeing provisioning for a long time until it times out with "OS Provisioning Timed Out" message. In these situations, cloud-init should report failure to the Azure datasource endpoint indicating provisioning failure. The user will immediately see provisioning terminate, giving them a much better failure experience instead of pointlessly waiting for OS provisioning timeout.
2020-11-18test_persistence: simplify VersionIsPoppedFromState (#674)Daniel Watkins
2020-11-18only run a subset of integration tests in CI (#672)Daniel Watkins
This introduces the "ci" mark, used to indicate a test which should run as part of our CI integration testing run and the integration-tests-ci tox environment, which runs only those tests. Travis has been adjusted to use this tox environment. (All current module tests have been marked with the "ci" mark, but the one bug test that we have has not.)
2020-11-18cli: add --system param to allow validating system user-data on a machine (#575)Chad Smith
Allow root user to validate the userdata provided to the launched machine using `cloud-init devel schema --system`
2020-11-17test_persistence: add VersionIsPoppedFromState test (#673)Daniel Watkins
2020-11-17introduce an upgrade framework and related testing (#659)Daniel Watkins
This commit does the following: * introduces the `cloudinit.persistence` module, containing `CloudInitPickleMixin` which provides lightweight versioning of objects' pickled representations (and associated testing) * introduces a basic upgrade testing framework (in `cloudinit.tests.test_upgrade`) which unpickles pickles from previous versions of cloud-init (stored in `tests/data/old_pickles`) and tests invariants that the current cloud-init codebase expects * uses the versioning framework to address an upgrade issue where `Distro.networking` could get into an unexpected state, and uses the upgrade testing framework to confirm that the issue is addressed
2020-11-17add --no-tty option to gpg (#669)Till Riedel
Make sure that gpg works even if the instance has no /dev/tty. This has been observed on Debian. LP: #1813396
2020-11-16Pin pycloudlib to a working commit (#666)James Falcon
2020-11-13DataSourceOpenNebula: exclude SRANDOM from context output (#665)Daniel Watkins
This is a new builtin variable that appeared in Ubuntu in 5.1~rc2-1ubuntu1 and started causing daily build failures.
2020-11-13cloud_tests: add hirsute release definition (#662)Daniel Watkins
2020-11-10split integration and cloud_tests requirements (#652)Daniel Watkins
The cloud_tests and the integration_tests (via pycloudlib) have conflicting requirements. This commit splits up their requirements files, so we can accurately express the requirements for each.
2020-11-10faq.rst: add warning to answer that suggests running `clean` (#661)Daniel Watkins
2020-11-10Fix stacktrace in DataSourceRbxCloud if no metadata disk is found (#632)Scott Moser
Largely speaking, ds-identify protects from this scenario being hit, but if DataSourceRbxCloud ran and there was no metadata disks found (LABEL=CLOUDMD), then it would stacktrace. The fix is just to clean up the get_md function a little bit, and the explicitly check for False as a return value in _get_data.
2020-11-09Make wakeonlan Network Config v2 setting actually work (#626)dermotbradley
Add code so that specifying "wakeonlan: true" actually results in relevant configuration entry appearing in /etc/network/interfaces, Netplan, and sysconfig for RHEL and OpenSuse. Add testcases for the above.
2020-11-06HACKING.md: unify network-refactoring namespace (#658)Mina Galić
We had some inconsistencies between `distro.net.<func>` and `distro.networking.<func>`. This commit unifies all such instances to `distro.networking.<func>`.
2020-11-06replace usage of dmidecode with kenv on FreeBSD (#621)Mina Galić
FreeBSD lets us read out kernel parameters with kenv(1), a user-space utility that's shipped in "base" We can use it in place of dmidecode(8), thus removing the dependency on sysutils/dmidecode, and the restrictions to i386 and x86_64 architectures that this utility imposes on FreeBSD. Co-authored-by: Scott Moser <smoser@brickies.net>
2020-11-05Prevent timeout on travis integration tests. (#651)James Falcon
Add a script in travis to output a dot every 10 minute during and integration test run to prevent travis timeout. Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-11-04azure: enable pushing the log to KVP from the last pushed byte (#614)Moustafa Moustafa
This allows the cloud-init log to be pushed multiple times during boot, with the latest lines being pushed each time.
2020-11-04Fix launch_kwargs bug in integration tests (#654)James Falcon
Integration test instance launch would previously fail if provided launch_kwargs is None
2020-11-04split read_fs_info into linux & freebsd parts (#625)Mina Galić
FreeBSD doesn't have blkid, so we want to use geom to list devices and their fstypes and labels. This PR also adds `jail` to the list of is_container() And we now also properly cache geom and blkid output! A test is added to verify the new behaviour by correctly identifying NoCloud on FreeBSD. Co-authored-by: Scott Moser <smoser@brickies.net>
2020-11-04PULL_REQUEST_TEMPLATE.md: expand commit message section (#642)Daniel Watkins
Make it a little easier for people to understand what we're looking for in a commit message.
2020-11-03Make some language improvements in growpart documentation (#649)Shane Frasier
* Fix awkward English in sentence * Add the missing word "the" * Fix misspelling * Add @jsf9k as a contributor Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-11-03Revert ".travis.yml: use a known-working version of lxd (#643)" (#650)Daniel Watkins
This reverts commit f72d0cb7a03cc311700ea8795edf2454172c7414. A new release of lxd has been released to the stable channel, which includes a fix for the issue we were previously working around.
2020-11-03Fix not sourcing default 50-cloud-init ENI file on Debian (#598)WebSpider
* Include both Ubuntu-style cfg file, and Debian-style directory in ENI * Add WebSpider as contributor
2020-11-02remove unnecessary reboot from gpart resize (#646)Mina Galić
a reboot after `gpart resize` hasn't been necessary since ca FreeBSD 9.2 Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-11-02cloudinit: move dmi functions out of util (#622)Scott Moser
This just separates the reading of dmi values into its own file. Some things of note: * left import of util in dmi.py only for 'is_container' It'd be good if is_container was not in util. * just the use of 'util.is_x86' to dmi.py * open() is used directly rather than load_file.
2020-11-02integration_tests: various launch improvements (#638)Daniel Watkins
* integration_tests: fix passing launch_kwargs to session_cloud.launch * integration_tests: log the launch_kwargs before launching instances * integration_tests: add support for specifying instance name for tests Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-11-02test_lp1886531: don't assume /etc/fstab exists (#639)Daniel Watkins
As the bug manifested because groovy doesn't ship /etc/fstab, we should not assume that /etc/fstab will be present for us to remove in our bootcmd. Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-11-02Remove Ubuntu restriction from PR template (#648)James Falcon
2020-11-02util: fix mounting of vfat on *BSD (#637)Mina Galić
Fix mounting of vfat filesystems by normalizing the different names for vfat to "msdos" which works across BSDs.
2020-11-02conftest: improve docstring for disable_subp_usage (#644)Daniel Watkins
Specifically by fixing references to `util`, adding some missing formatting, and adding information about a corner case of mock usage that `disable_subp_usage` breaks.
2020-11-02doc: add example query commands to debug Jinja templates (#645)Chad Smith
2020-10-30Correct documentation and testcase data for some user-data YAML (#618)dermotbradley
For cc_users_groups the user setting "expiredate" must be quoted in order for the relevant flag and value to be then passed to the useradd command. It its vaiue is not quoted then it is treated as Python type datetime.date and in `cloudinit/distros/__init__.py` the below "is it a string" condition fails and so no "--expiredate" parameter is passed to useradd and therefore it has no effect: ``` if key in useradd_opts and val and isinstance(val, str): useradd_cmd.extend([useradd_opts[key], val]) ``` For cc_users_groups, the user setting "inactive" does not actually disable accounts, the useradd "--inactive" option actually defines the number of days after password expiry that users can still login. So I have changed the docs to show it taking a quoted value of days (which works with the current code) rather than a boolean value. The quotes are necessary, like expiredate above, so that the value is also passed to the useradd command. For cc_power_state_change.py the "delay" setting value needs to have quotes around it as otherwise its leading plus sign will be stripped off.
2020-10-30Hetzner: Fix instance_id / SMBIOS serial comparison (#640)Markus Schade
Fixes erroneous string/int comparison introduced in 1431c8a metadata['instance-id'] is an integer but the value read from smbios is a string. The comparision would cause TypeError.