summaryrefslogtreecommitdiff
path: root/cloudinit
AgeCommit message (Collapse)Author
2016-12-20pep8: whitespace fixScott Moser
2016-12-20Update the list of valid ssh keys.Michael Felt
Update ssh_util.py with latest list of keys (from openssh-7.3p1/sshkeys.c), and remove extinct keys ending with "-v00@openssh.com" Added keys: rsa-sha2-256, rsa-sha2-512, ed25519, ssh-ed25519, ssh-ed25519-cert-v01@openssh.com Removed both of the double entries for the keys: ssh-dss-cert-v00@openssh.com ssh-rsa-cert-v00@openssh.com
2016-12-19set_hostname: avoid erroneously appending domain to fqdnLars Kellogg-Stedman
In some situations, cloud-init will erroneously append a default domain to an already fully qualified hostname, resulting in something like 'localhost.localdomain.localdomain'. This patch checks to see if the value returned by util.get_hostname() contains a '.', and if it does treats it as a fully qualified name. Resolves: rhbz#1389048 LP: #1647910
2016-12-19doc: change 'nobootwait' to 'nofail' in docsAnhad Jai Singh
'nobootwait' is an upstart specific extension to the mount syntax that is not supported by other mount systems. As Ubuntu 16.04 moved from upstart to systemd, support for 'nobootwait' was lost. All examples using 'nobootwait' are updated to use the standard 'nofail', which gives the expected behaviour of not failing to boot in case a volume is missing. There are subtle differences in semantics between 'nobootwait' and 'nofail', but it is the best substitute that gives behaviour similar to the upstart specific option.
2016-12-19Replace an expired bit.ly link in code comment.Joshua Harlow
The bit.ly link seems to have expired. Replace it with link to 'latest' version of EC2 docs.
2016-12-19user-groups: fix bug when groups was provided as string and had spacesScott Moser
Cloud-config provided like: users: - default - name: foobar groups: sudo, adm Would result in adduser being called as: useradd foobar --groups 'sudo, adm' -m Which would cause error: useradd: group ' adm' does not exist The fix here is just to always normalize groups and remove whitespace. Additionally a fix and unit tests to explicitly set system=False or no_create_home=True. Previously those paths did not test the value of the entry, only the presense of the entry. LP: #1354694
2016-12-11mounts: use mount -a again to accomplish mountsScott Moser
During recent changes to cc_mounts, on systemd systems, we started using systemctl daemon-reload rather than 'mount -a' to get mounts done. The belief was that since entries in /etc/fstab would be written to tell systemd that they should be after cloud-init (x-systemd.requires=cloud-init.service) that the reload would then let systemd mount the units as expected. That doesn't seem to work, and new mount entries end up not getting mounted. The change here moves back to using 'mount -a', but then also does a systemctl daemon-reload. LP: #1647708
2016-12-11CloudSigma: Fix bug where datasource was not loaded in local search.Scott Moser
CloudSigma would not get any datasources loaded during cloud-init local. Thus, when the network datasource was removed, *no* CloudSigma datasources would be loaded. LP: #1648380
2016-12-03when adding a user, strip whitespace from group listLars Kellogg-Stedman
The documentation shows group names in the 'groups:' key delimited by ", ", but this will result in group names that contain spaces. This can cause the 'groupadd' or 'useradd' commands to fail. This patch ensures that we strip whitespace from either end of the group names passed to the 'groups:' key. LP: #1354694
2016-12-02Replace usage of sys_netdev_info with read_sys_netJoshua Harlow
I've seen cases of unable to read from files as well as the existing os errors so catch io error and skip by using the smarter read_sys_net instead. LP: #1625766
2016-12-02fix problems found in python2.6 test.Joshua Harlow
These are just simple syntax fixes to work correctly on python2.6. Found when testing in a centos 6 container.
2016-11-28OpenStack: extend physical types to include hyperv, hw_veb, vhost_user.Scott Moser
This extends the list of device 'types' that are considered to be physical to include hyperv, hw_veb, and vhost_user. LP: #1642679
2016-11-22net/cmdline: Consider ip= or ip6= on command line not only ip=Scott Moser
The previous behavior would miss ip6= on the command line and would not pay attention to the written net-* or net6-* files if only ip6= was found. The fix here enables parsing the files if either ip= or ip6= is found, and adds some tests as well. LP: #1639930
2016-11-22Improve formatting for ProcessExecutionErrorWesley Wiedenmeier
This replaces long single lines in a log or console output with multiple lines that are much easier to read. It indents the stdout and stderr so logs are more easily read also.
2016-11-22Doc: various documentation fixesSean Bright
Several various minor fixes for the readthedocs documentation.
2016-11-22cloudinit/config/cc_rh_subscription.py: Remove repos before addingBrent Baude
A user has pointed out that upon set up of a machine, users typically remove repos (sometimes all of them) and then add repos in. This does make sense for a typical user.
2016-11-22main: set TZ in environment if not already set.Ryan Harper
If TZ is not set in environment, then datetime.datetime() will stat /etc/localtime on every usage. datetime is used in the logging path of cloud-init, and as such this is very busy. During a normal cloud-init boot, it results in we roughly 400+ stat calls to /etc/localtime. The value we set is :/etc/localtime, which is described at https://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html See also merge proposal at 307722 for more background. https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/307722
2016-11-22Azure: No longer rely on walinux agent.Scott Moser
Cloud-init has for some time relied on walinuxagent to do some bits of work necessary for instance initialization. That reliance has not been needed for a while, but we have still defaulted to it. This change uses the "builtin" path that Daniel Watkins added some time ago by default. Also, Adjust tests that assumed the non-__builtin__ Azure agent_command. LP: #1538522
2016-11-22disk_setup: Use sectors as unit when formatting MBR disks with sfdisk.Daniel Watkins
The version of sfdisk in wily (and onwards) only accepts sectors as a valid disk size. As such, this refactors the MBR code path in cc_disk_setup to use sectors. - use --unit=S: while newer versions of sfdisk assume --unit=S, older versions do not so we specifically pass it in. Versions of sfdisk found in supported OSes such as centos6 wont assume --unit=S. - add --force: this exists back to centos 6 (2.17.2), so it should be fine, and is what we ultimately want. "do what I say, even if it is stupid" - keep --Linux. Even though this has been deprecated for quite some time, we keep it until versions that want it are unsupported. If necessary at some point we could check for util linux version and if it had --Linux and use it in those cases. Additionally, improve usefulness of some log messages. LP: #1460715
2016-11-18Add activate_datasource, for datasource specific code paths.Scott Moser
This adds a call to 'activate_datasource'. That will be called during init stage (or init-local in the event of a 'local' dsmode). It is present so that the datasource can do platform specific operations that may be necessary. It is passed the fully rendered cloud-config and whether or not the instance is a new instance. The Azure datasource uses this to address formatting of the ephemeral devices. It does so by a.) waiting for the device to come online b.) removing the marker files for the disk_setup and mounts modules if it finds that the ephemeral device has been reset. LP: #1611074
2016-11-10pep8: fix style errors reported by pycodestyle 2.1.0Scott Moser
pycodestyle 2.1.0 is in Ubuntu zesty, and complained about the changes made here. Simple style changes. This makes 'make pep8' pass again when built in a zesty build system with proposed enabled.
2016-11-09Ec2: fix syntax and tox in previous commit.Scott Moser
Simply fix a commit that should not have been pushed.
2016-11-08Ec2: protect against non-dictionary in block-device-mapping.Scott Moser
Oracle public cloud has the string 'unavailable' in its metadata service for 'block-device-mapping'. The change here is to return None in device_name_to_device if that is the case.
2016-11-07Doc: fix spelling / typos in ca_certs and scripts_vendor.Scott Moser
Simple typo fixes.
2016-11-03net/cmdline: Further adjustments to ipv6 supportLaMont Jones
The implementation to add ipv6 support to Ubuntu initramfs changed (see bug 1621507). The changes here adjust to handle the new path. Now, the ipv6 route includes using the variable 'DEVICE6' in net6-DEVICE.conf files. LP: #1621615
2016-11-01dmidecode: Allow dmidecode to be used on aarch64Robert Schweikert
aarch64 systems have functional dmidecode, so allow that to be used. - aarch64 has support for dmidecode as well
2016-11-01AliYun: Add new datasource for Ali-Cloud ECSkaihuan.pkh
Support AliYun(Ali-Cloud ECS). This datasource inherits from EC2, the main difference is the meta-server address is changed to 100.100.100.200. The datasource behaves similarly to EC2 and relies on network polling. As such, it is not enabled by default.
2016-10-25cc_users_groups: fix remaing call to ds.normalize_user_groupsRyan Harper
Previous commit f0747c4b4cf073273e11d383f0354257be7276ed relocated normalize_users_groups to ug_util module, but missed modifying the cc_users_groups config module. Apply the location change to fix.
2016-10-25disk-config: udev settle after partitioning in gpt format.Scott Moser
The function exec_mkpart_gpt was simply not waiting for udev events to flush after calling sgdisk. The corresponding function exec_mkpart_mbr already did. This should fix a transient failure where mkfs would fail with 'not a block device'. LP: #1626243
2016-10-20Add documentation for logging features.Wesley Wiedenmeier
Update the summary of rsyslog module and add logging.rst to docs.
2016-10-20Add support for snap create-user on Ubuntu Core images.Ryan Harper
Ubuntu Core images use the `snap create-user` to add users to an Ubuntu Core system. Add support for creating snap users by adding a key to the users dictionary. users: - name: bob snapuser: bob@bobcom.io Or via the 'snappy' dictionary: snappy: email: bob@bobcom.io Users may also create a snap user without contacting the SSO by providing a 'system-user' assertion by importing them into snapd. Additionally, Ubuntu Core systems have a read-only /etc/passwd such that the normal useradd/groupadd commands do not function without an additional flag, '--extrausers', which redirects the pwd to /var/lib/extrausers. Move the system_is_snappy() check from cc_snappy module to util for re-use and then update the Distro class to append '--extrausers' if the system is Ubuntu Core.
2016-10-19Fix sshd restarts for rhel distros.Jim Gorz
Set the default value for 'ssh_svcname' in rhel distros. This means that it no longer needs to be set in system_info config.
2016-10-19OpenNebula: replace 'ip' parsing with cloudinit.net usage.Scott Moser
Replace the parsing of 'ip' to get a link and mac address list in OpenNebula's datasource with usage of cloudinit.net. This makes test cases there not depend on 'ip' availability and also uses common code.
2016-10-19Fix python2.6 things found running in centos 6.Scott Moser
This gets the tests running in centos 6. * ProcessExecutionError: remove setting of .message Nothing in cloud-init seems to use .message anywhere, so it does not seem necessary. The reason to change it is that on 2.6 it spits out: cloudinit/util.py:286: DeprecationWarning: BaseException.message * tox.ini: add a centos6 environment the tox versions listed here replicate a centos6 install with packages from EPEL. You will still need a python2.6 to run this env so we do not enable it by default.
2016-10-19Move user/group functions to new ug_util fileJoshua Harlow
The amount of code to do user and group normalization and extraction deserves its own file so move the code that does this to a new file and update references to the old location. This removes some of the funkyness done in config modules to avoid namespace and attribute clashes as well.
2016-10-18DigitalOcean: enable usage of data source by default.Scott Moser
Just add DigitalOcean to the list of datasources that are used if there is no 'datasource_list' provided in config.
2016-10-07MAAS: improve the main of datasource to look at kernel cmdline config.Scott Moser
This just looks in one other maas related path for a config file. The file '91_kernel_cmdline_url' is written by cloud-init when it gets a cloud-config-url parameter. Also now we read the config even if a url is specified to potentially fill in credentials.
2016-09-30Improve module documentation and doc cleanup.Wesley Wiedenmeier
This adds lots of config module documentation in a standard format. It will greatly improve the content at readthedocs. Additionally: * Add a 'doc' env to tox.ini * Changed default highlight language for sphinx conf from python to yaml most examples in documentation are yaml configs * Updated datasource examples to highlight sh code properly
2016-09-29lxd: Update network config for LXD 2.3Stéphane Graber
Prior to LXD 2.3, the bridge configuration was done through distro packaging. Thus, lxd module interacted with debconf. With 2.3 and higher, this is now done inside LXD itself, so we need to use "lxc network" there. For now, this perfectly matches what we had before with debconf and doesn't cover any of the new options. We can always add those later. A set of tests similar to what we had for debconf has been added to make sure things look good. This is tested in Yakkety container running LXD 2.3 and all options seem to be passed through as expected, giving me the bridge I defined. Signed-off-by: Stéphane Graber <stgraber@ubuntu.com>
2016-09-29DigitalOcean: use meta-data for network configruationBen Howard
On DigitalOcean, Network information is provided via Meta-data. It changes the datasource to be a local datasource, meaning it will run before fallback networking is configured. The advantage of that is that before networking is configured it can bring up a network device with ipv4 link-local and hit the metadata service that lives at 169.254.169.254 to find its networking configuration. It then takes down the link local address and lets cloud-init configure networking. The configuring of a network device to go looking for a metadata service is gated by a check of data in the smbios. This guarantees that the code will not run on another system.
2016-09-21Decode unicode types in decode_binaryRobert Schweikert
The test in decode_binary for six.text_type was incorrect as that includes unicode type in Python 2 which should actually be decoded. When the type is string_types we now properly check only for basestring and str in Python 2 and Python 3 respectively and return the given blob without making an attempt to decode.
2016-09-21Allow ephemeral drive to be unpartitionedPaul Meyer
If device has no partition table, the first line of output from `sgdisk -p <device>` will be "Creating new GPT entries.", instead of something like "Disk /dev/sdb: 266338304 sectors, 127.0 GiB". Also, protect against localized output by adjusting subp calls that parse sgdisk output to set LANG=C.
2016-09-21subp: add 'update_env' argumentScott Moser
In order for a caller to use 'env' argument of subp, they will realistically do: env = os.environ.copy() env['FOO'] = 'BZR' subp(cmd, env=env) This shortens that to be: subp(cmd, update_env={'FOO': 'BZR'}) Add tests, and update growpart tests to use mock when playing with os.environ.
2016-09-21net: support reading ipv6 dhcp config from initramfsLaMont Jones
This adds support for understanding 'dhcp6' as a protocol that can be written into /run/net-IFACE.cfg files by the initramfs. The end result is supporting ipv6 dhcp from initramfs boot all the way into iscsi root. LP: #1621615, #1621507
2016-09-20Adjust mounts and disk configuration for systemd.Scott Moser
The end result of all of these changes is to get mounts managed by cloud-init to occur only after cloud-init.service is done. We need to do that so that filesystems that are set up by cloud-init (in disk_setup) do not get mounted by stale entries in /etc/fstab before the setup occurs. This can occur in 2 ways: a.) new instance with old /etc/fstab b.) same instance where disk needs adjusting (Azure resize will re-format the ephemeral disk). The list of changes here is: - move mounts and disk_setup module to cloud-init.service rather than config. cloud-init.service runs earlier in boot so it can get those mount points done earlier. - on systemd add 'x-systemd.requires=cloud-init.service' to fstab options - cloud-init-local.service: add Before=basic.target - cloud-init.service: - extend After, Before, and Wants to multiple lines rather than one long line. - sort consistently with cloud-init-local.service - add DefaultDependencies=no - add Before=default.target - add Conflicts=shutdown.target LP: #1611074
2016-09-15dmidecode: run dmidecode only on i?86 or x86_64 arch.Robert Schweikert
Dmidecode is not going to run successfully on anything other than an x86 or x86_64, just avoid running it anywhere else.
2016-09-12release 0.7.8Scott Moser
Bump the version in cloudinit/version.py to be 0.7.8.
2016-09-12Allow link type of null in network_data.jsonJon Grimm
Treat null type as yet another physical type, seen in real-world openstack cloud. Also, support the case where network_data.json provides mac addresses in upper case. Rackspace public cloud currently does that. LP: #1621968
2016-09-12DataSourceOVF: fix user-data as base64 with python3Scott Moser
When user-data was provided in the ovf environment python3 would call base64.decodestring() with a string rather than bytes and an exception would occur. This fixes the broken path and adds unit test. Also changes to return None rather than empty string when there is no user-data and when there is user-data return that as bytes instead of string. LP: #1619394
2016-09-08apt config conversion: treat empty string as not provided.Scott Moser
Old behavior allowed a user to provide: apt_mirror: "" And that was the same as: apt_mirror: null and the same as having not specified apt_mirror at all. This maintains that behavior for all old string values. LP: #1621180