summaryrefslogtreecommitdiff
path: root/cloudinit
AgeCommit message (Collapse)Author
2020-08-13Fix 'Users and Groups' configuration documentation (#530)sshedi
Few of the 'User and Groups' configurations in cloud-config have no effect on already existing users. This was not documented earlier. This change set adds that information to documentation. Signed-off-by: Shreenidhi Shedi <sshedi@vmware.com>
2020-08-13cloudinit.distros: update docstrings of add_user and create_user (#527)Daniel Watkins
This aligns their docstrings more closely with their actual behaviour.
2020-08-10DataSourceOracle: refactor to use only OPC v1 endpoint (#493)Daniel Watkins
The /opc/v1/ metadata endpoints[0] are universally available in Oracle Cloud Infrastructure and the OpenStack endpoints are considered deprecated, so we can refactor the data source to use the OPC endpoints exclusively. This simplifies the datasource code substantially, and enables use of OPC-specific attributes in future. [0] https://docs.cloud.oracle.com/en-us/iaas/Content/Compute/Tasks/gettingmetadata.htm
2020-08-10Fix a typo in apt pipelining module (#525)Xiao Liang
* Fix a typo in apt pipelining module Changed `whcih` to `which`. * Update .github-cla-signers I have signed the CLA on Canonical's site, adding my username to list of CLA signers. * Update .github-cla-signers I need to sort the list alphabetically.
2020-08-07Recognize LABEL_FATBOOT labels (#513)James Falcon
Update DataSourceNoCloud and ds-identify to recognize LABEL_FATBOOT labels from blkid. Also updated associated tests. LP: #1841466
2020-08-06Handle additional identifier for SLES For HPC (#520)Robert Schweikert
Add "sle_hpc" to list of values which are variant 'suse'.
2020-07-22azure: disable bouncing hostname when setting hostname fails (#494)Anh Vo
DataSourceAzure: Gracefully handle the case of set hostname failure during provisioning
2020-07-21VMware: Support parsing DEFAULT-RUN-POST-CUST-SCRIPT (#441)xiaofengw-vmware
Add support for VMware's vCD configuration setting DEFAULT-RUN-POST-CUST-SCRIPT. When set True, it will default vms to run post customization scripts if the VM has not been configured in VMTools with "enable-custom-scripts" set False. Add datasource documentation with a bit more context about this interaction on VMware products. With this fix, the behavior will be: * If VM administrator doesn't want others to execute a script on this VM, VMtools can set "enable-custom-scripts" to false from the utility "vmware-toolbox-cmd". * If VM administrator doesn't set value to "enable-custom-scripts", then by default this script is disabled for security purpose. * For VMware's vCD product , the preference is to enable the script if "enable-custom-scripts" is not set. vCD will generate a configuration file with "DEFAULT-RUN-POST-CUST-SCRIPT" set to true. This flag works for both VMware customization engine and cloud-init.
2020-07-15DataSourceAzure: Use ValueError when JSONDecodeError is not available (#490)Anh Vo
JSONDecodeError is only available in Python 3.5+. When it isn't available (i.e. on Python 3.4, which cloud-init still supports) use the more generic ValueError.
2020-07-15cc_ca_certs.py: fix blank line problem when removing CAs and adding new one ↵dermotbradley
(#483) Problem: When cc_ca_certs configuration has both "remove-defaults: true" and also specifies one, or more, new trusted CAs to add then the resultant /etc/ca-certificates.conf file's 1st line is blank. As noted in comments in the existing cc_ca_certs.py code blank lines in this file cause problems. Fix: Before adding the cloud-init CA filename to this file first check the size of the file - if is is empty (as all existing CAs have been deleted) then write only the cloud-init CA filename to the file rather than appending it to the file.
2020-07-15ssh exit with non-zero status on disabled user (#472)Eduardo Otubo
It is confusing for scripts, where a disabled user has been specified, that ssh exits with a zero status by default without indication anything failed. I think exitting with a non-zero status would make more clear in scripts and automated setups where things failed, thus making noticing the issue and debugging easier. Signed-off-by: Eduardo Otubo <otubo@redhat.com> Signed-off-by: Aleksandar Kostadinov <akostadi@redhat.com> LP: #1170059
2020-07-15cloudinit: remove global disable of pylint W0107 and fix errors (#489)Daniel Watkins
* cloudinit: remove global disable of pylint W0107 and fix errors This includes removing a test class which contained no tests but wasn't detected as empty because of an errant pass statement. * .pylintrc: update disable comment to match arguments
2020-07-14networking: refactor wait_for_physdevs from cloudinit.net (#466)Daniel Watkins
* Refactor `cloudinit.net.wait_for_physdevs` to `cloudinit.distros.networking.Networking.wait_for_physdevs` * Split the Linux-specific `udevadm_settle` call out to a separate abstract `Networking.settle` method; implement it on `LinuxNetworking` and add a `NotImplementedError` implementation to `BSDNetworking` * Modify `wait_for_physdevs`s one callsite to use the new location LP: #1884626
2020-07-13cloudinit: remove global disable of pylint W0105 and fix errors (#480)Daniel Watkins
This includes a fix to a test that had a string concatenation issue, and so was only testing a prefix of what was intended.
2020-07-13Fix two minor warnings (#475)Daniel Watkins
2020-07-10cc_mounts: handle missing fstab (#484)Ryan Harper
Do not fail if /etc/fstab is not present. Some images, like container rootfs may not include this file by default. LP: #1886531
2020-07-06cloudinit: fix tip-pylint failures and bump pinned pylint version (#478)Daniel Watkins
Specifically: * disable E1102 in cloudinit/sources/helpers/openstack.py for reasons described in a comment, and * refactor `abs_join` to require at least one positional argument; this matches os.path.join's signature, and that mismatch is what was causing pylint to emit a warning * bump to pylint 2.4.2
2020-07-02tests: use markers to configure disable_subp_usage (#473)Daniel Watkins
This is an improvement over indirect parameterisation for a few reasons: * The test code is much easier to read, the mark names are much more intuitive than the indirect parameterisation invocation, and there's less boilerplate to boot * The fixture no longer has to overload the single parameter that fixtures can take with multiple meanings
2020-06-30Disable ec2 mirror for non aws instances (#390)lucasmoura
For versions before 20.2, we allowed the use of ec2 mirrors if the datasource availability_zone matches one of the ec2 regions. We are now updating that behavior to allow allow the use of ec2 mirrors on ec2 instances or if the user directly passes an an ec2 mirror url through #cloud-config apt directives. LP: #1456277
2020-06-30networking: refactor is_physical from cloudinit.net (#457)Daniel Watkins
As the first refactor PR, this also includes the initial structure for tests. LP: #1884619
2020-06-30Enable use of the caplog fixture in pytest tests, and add a cc_final_message ↵Daniel Watkins
test using it (#461) caplog is only available in pytest itself from 3.0 onwards. In xenial, we only have pytest 2.8.7. However, in xenial we do have pytest-catchlog available (as python3-pytest-catchlog), so we use that where appropriate.
2020-06-29RbxCloud: Add support for FreeBSD (#464)Adam Dobrawy
Changes are made that simplify code and aim to properly support FreeBSD: - use `util.find_devs_with` instead call directly `blkid`, because on FreeBSD is not supported well and `util.find_devs_with` have solution for FreeBSD for that - introduction of an additional name on FAT file system, which is used in FreeBSD - drop shell to use default value, because FreeBSD – by default – does not have `/bin/bash`
2020-06-29Add schema for cc_chef module (#375)lucasmoura
Create a schema object for the chef module and validate this schema in the handle function of the module. Some of the config keys description, so I tried looking at the code and chef documentation to provide an information to the user. However, I don't know if I have the best description for all fields. For example, for the key show_time I could not find an accurate description of what it did, so I used what was in our code base to infer what it should do. LP: #1858888
2020-06-29test_util: add (partial) testing for util.mount_cb (#463)Daniel Watkins
2020-06-24net/networking: remove unused functions/methods (#453)Daniel Watkins
Namely, is_connected, is_wireless and is_present. None of these are used in the cloud-init codebase, so remove the dead code (instead of refactoring it).
2020-06-23distros.networking: initial implementation of layout (#391)Daniel Watkins
This commit introduces the initial structure for the "cloudinit.net -> cloudinit.distros.networking Hierarchy" refactor, as detailed in [0]. It also updates that section with some changes driven by this initial implementation, as well as adding a lot more specifics to it. [0] https://cloudinit.readthedocs.io/en/latest/topics/hacking.html#cloudinit-net-cloudinit-distros-networking-hierarchy
2020-06-22Hetzner: support reading user-data that is base64 encoded. (#448)Scott Moser
Hetzner cloud only supports user-data as a string (presumably utf-8). In order to allow users on Hetzner to provide binary data to cloud-init, we will attempt to base64decode the userdata. The change here adds a 'maybe_b64decode' function that will decode data if and only if is base64 encoded. The reason for not using util.b64d is that we do not want the return value decoded to a string, and util.b64d will do that if it can. Additionally we call decode with validate=True which oddly is not the default. LP: #1884071
2020-06-22cc_final_message: don't create directories when writing boot-finished (#445)Daniel Watkins
If the instance symlink doesn't exist, then we shouldn't create a directory in its place, because that breaks future boots. LP: #1883903
2020-06-19util: add ensure_dir_exists parameter to write_file (#443)Daniel Watkins
This allows us to disable the `ensure_dir` call when it isn't appropriate.
2020-06-19printing the error stream of the dhclient process before killing it (#369)Moustafa Moustafa
This introduces a way to log the dhclient error stream, and uses it for the Azure datasource (where we have a specific requirement for this data to be logged).
2020-06-17util: rename write_file's copy_mode parameter to preserve_mode (#439)Daniel Watkins
When updating the docstring to include it, I realised that the current name is somewhat misleading; this makes it a little easier to understand, I think.
2020-06-10test: fix all flake8 E126 errors (#425)Joshua Powers
2020-06-09Fixes KeyError for bridge with no "parameters:" setting (#423)Brian Candler
Reason: commit ded1ec8 introduced a regression whereby a bridge with no "parameters:" setting caused a KeyError exception. LP: #1879673
2020-06-09When tools.conf does not exist, running cmd "vmware-toolbox-cmd config get ↵chengcheng-chcheng
deployPkg enable-custom-scripts", the return code will be EX_UNAVAILABLE(69), on this condition, it should not take it as error. (#413)
2020-06-08test: move conftest.py to top-level, to cover tests/ also (#414)Daniel Watkins
* test_opennebula: convert TestParseShellConfig to a pytest test And allow it to run bash. (We aren't aiming to convert TestCase tests to pytest tests as a rule. In this case, I needed to change its implementation to limit subp usage, and I chose pytest over CiTestCase.) * test: move conftest.py to top-level, to cover tests/ also This gives us a single conftest.py which is shared by all tests in the project.
2020-06-08Replace cc_chef is_installed with use of subp.is_exe. (#421)Scott Moser
This was brought up in review of #416. Makes sense to remove the local copy of "is this executable file".
2020-06-08Move runparts to subp. (#420)Scott Moser
runparts (run a directory of scripts) seems to fit well in subp module. The request to move it there was raised in #416. Replace use of logexc with LOG.debug as logexc comes from util.
2020-06-08Move subp into its own module. (#416)Scott Moser
This was painful, but it finishes a TODO from cloudinit/subp.py. It moves the following from util to subp: ProcessExecutionError subp which target_path I moved subp_blob_in_tempfile into cc_chef, which is its only caller. That saved us from having to deal with it using write_file and temp_utils from subp (which does not import any cloudinit things now). It is arguable that 'target_path' could be moved to a 'path_utils' or something, but in order to use it from subp and also from utils, we had to get it out of utils.
2020-06-04New feature flag functionality and fix includes failing silently (#367)James Falcon
Build time feature flags are now defined in cloudinit/features.py. Feature flags can be added to toggle configuration options or deprecated features. Feature flag overrides can be placed in cloudinit/feature_overrides.py. Further documentation can be found in HACKING.rst. Additionally, updated default behavior to exit with an exception if #include can't retrieve resources as expected. This behavior can be toggled with a feature flag. LP: #1734939
2020-06-03Enhance poll imds logging (#365)Moustafa Moustafa
Improving the debugability of this code path by logging the thrown exception details for the non 404 exceptions. Retry IMDS on HTTP Error 404 and 410, re-run DHCP on other exceptions.
2020-06-02test: fix all flake8 E121 and E123 errors (#404)Joshua Powers
This fixes issues with closing brackets not matching the opening bracket's line and continuation line under-idented for hanging indent.
2020-06-02test: fix all flake8 E241 (#403)Joshua Powers
Remove extra spaces after a ','
2020-06-01cc_grub_dpkg: determine idevs in more robust manner with grub-probe (#358)Matthew Ruffell
Replace the hardcoded list of devices with a more robust way of determining the device which grub is installed to. We use grub-probe to fetch the underlying disk the /boot directory is located on, and attempt to match the disk with its /dev/disk/by-id value. If no such /dev/disk/by-id/ value exists, we fallback to the plain disk name. The changes are robust to unstable kernel device names and ordering, and use /dev/disk/by-id values to populate grub-pc/install_devices where possible. LP: #1877491
2020-06-01test: fix all flake8 E741 errors (#401)Joshua Powers
This removes the use of variables named ‘l’, ‘O’, or ‘I’. Generally these are used in list comprehension to read the line of lines.
2020-05-27Enable chef_license support for chef infra client (#389)Bipin Bachhao
Co-authored-by: Daniel Watkins <oddbloke@ubuntu.com>
2020-05-21cc_snap: validate that assertions property values are strings (#370)Daniel Watkins
And add an example of providing a list of assertions.
2020-05-21conftest: implement partial disable_subp_usage (#371)Daniel Watkins
This allows tests to be configured to permit some commands to be run via util.subp, while still rejecting any unexpected calls. See the documentation for further details.
2020-05-19test_resolv_conf: refresh stale comment (#374)Daniel Watkins
2020-05-18cc_snap: apply validation to snap.commands properties (#364)Daniel Watkins
Specifically, ensure that given values are either strings, or arrays of strings.
2020-05-18make finding libc platform independent (#366)Mina Galić (deprecated: Igor Galić)
and slower. and since we're making it slower, let's cache it, in case boottime gets called more than once.