summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2020-11-02cloudinit: move dmi functions out of util (#622)Scott Moser
This just separates the reading of dmi values into its own file. Some things of note: * left import of util in dmi.py only for 'is_container' It'd be good if is_container was not in util. * just the use of 'util.is_x86' to dmi.py * open() is used directly rather than load_file.
2020-11-02integration_tests: various launch improvements (#638)Daniel Watkins
* integration_tests: fix passing launch_kwargs to session_cloud.launch * integration_tests: log the launch_kwargs before launching instances * integration_tests: add support for specifying instance name for tests Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-11-02test_lp1886531: don't assume /etc/fstab exists (#639)Daniel Watkins
As the bug manifested because groovy doesn't ship /etc/fstab, we should not assume that /etc/fstab will be present for us to remove in our bootcmd. Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-11-02Remove Ubuntu restriction from PR template (#648)James Falcon
2020-11-02util: fix mounting of vfat on *BSD (#637)Mina Galić
Fix mounting of vfat filesystems by normalizing the different names for vfat to "msdos" which works across BSDs.
2020-11-02conftest: improve docstring for disable_subp_usage (#644)Daniel Watkins
Specifically by fixing references to `util`, adding some missing formatting, and adding information about a corner case of mock usage that `disable_subp_usage` breaks.
2020-11-02doc: add example query commands to debug Jinja templates (#645)Chad Smith
2020-10-30Correct documentation and testcase data for some user-data YAML (#618)dermotbradley
For cc_users_groups the user setting "expiredate" must be quoted in order for the relevant flag and value to be then passed to the useradd command. It its vaiue is not quoted then it is treated as Python type datetime.date and in `cloudinit/distros/__init__.py` the below "is it a string" condition fails and so no "--expiredate" parameter is passed to useradd and therefore it has no effect: ``` if key in useradd_opts and val and isinstance(val, str): useradd_cmd.extend([useradd_opts[key], val]) ``` For cc_users_groups, the user setting "inactive" does not actually disable accounts, the useradd "--inactive" option actually defines the number of days after password expiry that users can still login. So I have changed the docs to show it taking a quoted value of days (which works with the current code) rather than a boolean value. The quotes are necessary, like expiredate above, so that the value is also passed to the useradd command. For cc_power_state_change.py the "delay" setting value needs to have quotes around it as otherwise its leading plus sign will be stripped off.
2020-10-30Hetzner: Fix instance_id / SMBIOS serial comparison (#640)Markus Schade
Fixes erroneous string/int comparison introduced in 1431c8a metadata['instance-id'] is an integer but the value read from smbios is a string. The comparision would cause TypeError.
2020-10-30.travis.yml: use a known-working version of lxd (#643)Daniel Watkins
The latest lxd 4.7 snap causes failures; while we're diagnosing those and addressing the issues (either upstream or in our code), pin at lxd 4.6 so we can continue to land PRs in cloud-init trunk.
2020-10-29tools/build-on-freebsd: fix comment explaining purpose of the script (#635)Mina Galić
Since there is now an official port for cloud-init in FreeBSD, this script has a different purpose. It's an interface between cloud-init maintainers, and the FreeBSD port maintainers for quickly seeing which dependencies have changed.
2020-10-29Hetzner: initialize instance_id from system-serial-number (#630)Markus Schade
Hetzner Cloud also provides the instance ID in SMBIOS information. Use it to locally check_instance_id and to compared with instance_id from metadata service. LP: #1885527
2020-10-29Explicit set IPV6_AUTOCONF and IPV6_FORCE_ACCEPT_RA on static6 (#634)Eduardo Otubo
The static and static6 subnet types for network_data.json were being ignored by the Openstack handler, this would cause the code to break and not function properly. As of today, if a static6 configuration is chosen, the interface will still eventually be available to receive router advertisements or be set from NetworkManager to wait for them and cycle the interface in negative case. It is safe to assume that if the interface is manually configured to use static ipv6 address, there's no need to wait for router advertisements. This patch will set automatically IPV6_AUTOCONF and IPV6_FORCE_ACCEPT_RA both to "no" in this case. This patch fixes the specific behavior only for RHEL flavor and sysconfig renderer. It also introduces new unit tests for the specific case as well as adjusts some existent tests to be compatible with the new options. This patch also addresses this problem by assigning the appropriate subnet type for each case on the openstack handler. rhbz: #1889635 rhbz: #1889635 Signed-off-by: Eduardo Otubo otubo@redhat.com
2020-10-29get_interfaces: don't exclude Open vSwitch bridge/bond members (#608)Lukas Märdian
If an OVS bridge was used as the only/primary interface, the 'init' stage failed with a "Not all expected physical devices present" error, leaving the system with a broken SSH setup. LP: #1898997
2020-10-28Add config modules for controlling IBM PowerVM RMC. (#584)Aman306
Reliable Scalable Cluster Technology (RSCT) is a set of software components that together provide a comprehensive clustering environment(RAS features) for IBM PowerVM based virtual machines. RSCT includes the Resource Monitoring and Control (RMC) subsystem. RMC is a generalized framework used for managing, monitoring, and manipulating resources. RMC runs as a daemon process on individual machines and needs creation of unique node id and restarts during VM boot. LP: #1895979 Co-authored-by: Scott Moser <smoser@brickies.net>
2020-10-28lp-to-git-users: adding asciiprod (#629)Markus Schade
Mapped from lp-markusschade
2020-10-27Update network config docs to clarify MAC address quoting (#623)dermotbradley
Also update MAC addresses used in testcases to remove quotes where not required and add single quotes where quotes are required.
2020-10-27gentoo: fix hostname rendering when value has a comment (#611)Manuel Aguilera
Gentoo's hostname file format instead of being just the host name is hostname=thename". The old code works fine when the file has no comments but if there is a comment the line ``` gentoo_hostname_config = 'hostname="%s"' % conf ``` can render an invalid hostname file that looks similar to ``` hostname="#This is the host namehello" ``` The fix inserts the hostname in a gentoo friendly way so that it gets handled by HostnameConf as a whole and comments are handled and preserved
2020-10-26refactor integration testing infrastructure (#610)James Falcon
* Separated IntegrationClient into separate cloud and instance abstractions. This makes it easier to control the lifetime of the pycloudlib's cloud and instance abstractions separately. * Created new cloud-specific subclasses accordingly * Moved platform parsing and initialization code into its own file * Created new session-wide autorun fixture to automatically initialize and destroy the dynamic cloud
2020-10-23stages: don't reset permissions of cloud-init.log every boot (#624)Daniel Watkins
ensure_file needed modification to support doing this, so this commit also includes the following changes: test_util: add tests for util.ensure_file util: add preserve_mode parameter to ensure_file util: add (partial) type annotations to ensure_file LP: #1900837
2020-10-22docs: Add how to use cloud-localds to boot qemu (#617)Joshua Powers
* docs: Add hot to use cloud-localds to boot qemu There is a complete lack of documentation on using cloud-localds with cloud-init to boot an image locally. Drive by, added some more whitepapers, blogs, and videos * fix line length * * add where cloud-localds comes from * add more specific example with metadata and network config * Add link to cloud-utils package
2020-10-21Drop vestigial update_resolve_conf_file function (#620)Scott Moser
update_resolve_conf_file is no longer used. The last reference to it was removed in c3680475f9c970, which was itself a "remove dead code" commit.
2020-10-20cc_mounts: correctly fallback to dd if fallocate fails (#585)Daniel Watkins
`create_swap()` was previously catching and not re-raising the ProcessExecutionError that indicated swap creation failure; this meant that the fallback logic could never be triggered. This commit adds the required re-raise (as well as removing a duplicated log message). LP: #1897099
2020-10-20.travis.yml: add integration-tests to Travis matrix (#600)Daniel Watkins
This is implemented as a copy/paste of the `citest` integration testing script: the two are not intended to co-exist long-term, so it isn't worth further complicating an already complex part of our Travis configuration for short-term code reuse. The two changes from the `citest` definition: the test framework executed on the last line of `script`, and it is given a `name` so we can easily ignore failures.
2020-10-20ssh_util: handle non-default AuthorizedKeysFile config (#586)Eduardo Otubo
The following commit merged all ssh keys into a default user file `~/.ssh/authorized_keys` in sshd_config had multiple files configured for AuthorizedKeysFile: commit f1094b1a539044c0193165a41501480de0f8df14 Author: Eduardo Otubo <otubo@redhat.com> Date: Thu Dec 5 17:37:35 2019 +0100 Multiple file fix for AuthorizedKeysFile config (#60) This commit ignored the case when sshd_config would have a single file for AuthorizedKeysFile, but a non default configuration, for example `~/.ssh/authorized_keys_foobar`. In this case cloud-init would grab all keys from this file and write a new one, the default `~/.ssh/authorized_keys` causing the bug. rhbz: #1862967 Signed-off-by: Eduardo Otubo <otubo@redhat.com>
2020-10-19bddeb: new --packaging-branch argument to pull packaging from branch (#576)Paride Legovini
bddeb builds a .deb package using the template packaging files in packages/debian/. The new --packaging-branch flag allows to specify a git branch where to pull the packaging (i.e. the debian/ directory) from. This is useful to build a .deb package from master with the very same packaging which is used for the uploads.
2020-10-19Add more integration tests (#615)lucasmoura
Translate the following tests from `cloud_tests` to the new integration test framework: * test_runcmd.py * seed_random_data.py * set_hostname.py * set_hostname_fqdn.py * snap.py * ssh_auth_key_fingerprints_disable.py * ssh_auth_key_fingerprints_enable.py * ssh_import_id.py * ssh_keys_generate.py * ssh_keys_provided.py * timezone.py * write_files.py
2020-10-16DataSourceAzure: write marker file after report ready in preprovisioning (#590)Johnson Shi
DataSourceAzure previously writes the preprovisioning reported ready marker file before it goes through the report ready workflow. On certain VM instances, the marker file is successfully written but then reporting ready fails. Upon rare VM reboots by the platform, cloud-init sees that the report ready marker file already exists. The existence of this marker file tells cloud-init not to report ready again (because it mistakenly assumes that it already reported ready in preprovisioning). In this scenario, cloud-init instead erroneously takes the reprovisioning workflow instead of reporting ready again.
2020-10-16integration_tests: emit settings to log during setup (#601)Daniel Watkins
2020-10-16integration_tests: implement citest tests run in Travis (#605)Daniel Watkins
Specifically: * `apt_configure_sources_list` * `ntp_servers` * `set_password_list` * `users_groups` Although not currently run in Travis, `set_password_list_string` was ported over alongside `set_password_list` (as `test_set_password`).
2020-10-15Add Azure support to integration test framework (#604)James Falcon
2020-10-15openstack: consider product_name as valid chassis tag (#580)Adrian Vladu
Consider valid product names as valid chassis asset tags when detecting OpenStack platform before crawling for OpenStack metadata. As `ds-identify` tool uses product name as valid chassis asset tags, let's replicate the behaviour in the OpenStack platform detection too. This change should be backwards compatible and a temporary fix for the current limitations on the OpenStack platform detection. LP: #1895976
2020-10-15azure: clean up and refactor report_diagnostic_event (#563)Johnson Shi
This moves logging into `report_diagnostic_event`, to clean up its callsites.
2020-10-13net: add the ability to blacklist network interfaces based on driver during ↵Anh Vo
enumeration of physical network devices (#591)
2020-10-06integration_tests: don't error on cloud-init failure (#596)Daniel Watkins
pycloudlib's default behaviour is to raise an exception if cloud-init fails to run in an instance being launched. For cloud-init testing, we want our test assertions to flag up failures, so we disable this behaviour for instances we launch.
2020-10-06integration_tests: improve cloud-init.log assertions (#593)Daniel Watkins
2020-10-06conftest.py: remove top-level import of httpretty (#599)Daniel Watkins
This means that the integration tests do not need to install test-requirements.txt in order to successfully import `conftest.py`.
2020-10-05tox.ini: add integration-tests testenv definition (#595)Daniel Watkins
2020-10-05PULL_REQUEST_TEMPLATE.md: empty checkboxes need a space (#597)Daniel Watkins
2020-10-02add integration test for LP: #1886531 (#592)Daniel Watkins
2020-10-01Initial implementation of integration testing infrastructure (#581)James Falcon
2020-10-01Fix name of ntp and chrony service on CentOS and RHEL. (#589)Scott Moser
The service installed by the CentOS and RHEL 'ntp' package is ntpd.service not ntp.service Fix that for those two distros. Also fix chrony service from 'chrony' to 'chronyd'. LP: #1897915
2020-09-29Adding a PR template (#587)James Falcon
2020-09-24Azure parse_network_config uses fallback cfg when generate IMDS network cfg ↵Johnson Shi
fails (#549) Azure datasource's `parse_network_config` throws a fatal uncaught exception when an exception is raised during generation of network config from IMDS metadata. This happens when IMDS metadata is invalid/corrupted (such as when it is missing network or interface metadata). This causes the rest of provisioning to fail. This changes `parse_network_config` to be a non-fatal implementation. Additionally, when generating network config from IMDS metadata fails, fall back on generating fallback network config (`_generate_network_config_from_fallback_config`). This also changes fallback network config generation (`_generate_network_config_from_fallback_config`) to blacklist an additional driver: `mlx5_core`.
2020-09-23features: refresh docs for easier out-of-context reading (#582)Daniel Watkins
2020-09-22Fix typo in resolv_conf module's description (#578)Wacław Schiller
2020-09-22cc_users_groups: minor doc formatting fix (#577)Daniel Watkins
Co-authored-by: Rick Harding <rharding@mitechie.com>
2020-09-21Fix typo in disk_setup module's description (#579)Wacław Schiller
2020-09-18Add vendor-data support to seedfrom parameter for NoCloud and OVF (#570)Johann Queuniet
2020-09-16boot.rst: add First Boot Determination section (#568)Daniel Watkins
LP: #1888858