summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2016-12-21Fix config order of precedence, putting kernel command line over system.Wesley Wiedenmeier
The correct order of precedence when reading the base config: builtin config system config kernel command line provided config. This reverts commit 63501f44, which actually broke the behavior it reported to fix. It also adds some unit tests to ensure this behavior is not broken again. LP: #1582323
2016-12-20pep8: whitespace fixScott Moser
2016-12-20Update the list of valid ssh keys.Michael Felt
Update ssh_util.py with latest list of keys (from openssh-7.3p1/sshkeys.c), and remove extinct keys ending with "-v00@openssh.com" Added keys: rsa-sha2-256, rsa-sha2-512, ed25519, ssh-ed25519, ssh-ed25519-cert-v01@openssh.com Removed both of the double entries for the keys: ssh-dss-cert-v00@openssh.com ssh-rsa-cert-v00@openssh.com
2016-12-19network: add ENI unit test for statically rendered routes.Scott Moser
This just adds a unit test for a case found to be failing in curtin. The issue was reported under bug 1649652.
2016-12-19set_hostname: avoid erroneously appending domain to fqdnLars Kellogg-Stedman
In some situations, cloud-init will erroneously append a default domain to an already fully qualified hostname, resulting in something like 'localhost.localdomain.localdomain'. This patch checks to see if the value returned by util.get_hostname() contains a '.', and if it does treats it as a fully qualified name. Resolves: rhbz#1389048 LP: #1647910
2016-12-19doc: change 'nobootwait' to 'nofail' in docsAnhad Jai Singh
'nobootwait' is an upstart specific extension to the mount syntax that is not supported by other mount systems. As Ubuntu 16.04 moved from upstart to systemd, support for 'nobootwait' was lost. All examples using 'nobootwait' are updated to use the standard 'nofail', which gives the expected behaviour of not failing to boot in case a volume is missing. There are subtle differences in semantics between 'nobootwait' and 'nofail', but it is the best substitute that gives behaviour similar to the upstart specific option.
2016-12-19Replace an expired bit.ly link in code comment.Joshua Harlow
The bit.ly link seems to have expired. Replace it with link to 'latest' version of EC2 docs.
2016-12-19user-groups: fix bug when groups was provided as string and had spacesScott Moser
Cloud-config provided like: users: - default - name: foobar groups: sudo, adm Would result in adduser being called as: useradd foobar --groups 'sudo, adm' -m Which would cause error: useradd: group ' adm' does not exist The fix here is just to always normalize groups and remove whitespace. Additionally a fix and unit tests to explicitly set system=False or no_create_home=True. Previously those paths did not test the value of the entry, only the presense of the entry. LP: #1354694
2016-12-11mounts: use mount -a again to accomplish mountsScott Moser
During recent changes to cc_mounts, on systemd systems, we started using systemctl daemon-reload rather than 'mount -a' to get mounts done. The belief was that since entries in /etc/fstab would be written to tell systemd that they should be after cloud-init (x-systemd.requires=cloud-init.service) that the reload would then let systemd mount the units as expected. That doesn't seem to work, and new mount entries end up not getting mounted. The change here moves back to using 'mount -a', but then also does a systemctl daemon-reload. LP: #1647708
2016-12-11CloudSigma: Fix bug where datasource was not loaded in local search.Scott Moser
CloudSigma would not get any datasources loaded during cloud-init local. Thus, when the network datasource was removed, *no* CloudSigma datasources would be loaded. LP: #1648380
2016-12-03when adding a user, strip whitespace from group listLars Kellogg-Stedman
The documentation shows group names in the 'groups:' key delimited by ", ", but this will result in group names that contain spaces. This can cause the 'groupadd' or 'useradd' commands to fail. This patch ensures that we strip whitespace from either end of the group names passed to the 'groups:' key. LP: #1354694
2016-12-02fix decoding of utf-8 chars in yaml testScott Moser
Python 3 would fail to load yaml from doc/examples/cloud-config-apt.txt when the LANG (specifically LC_CTYPE) was 'C'. The changes here do 2 things: a.) remove the non-ascii characters from the yaml file. b.) fix the validate-yaml.py program to decode using utf-8 specifically rather than using the inherited settings. This fixes it now for ascii and in the future also should non-ascii slip in.
2016-12-02Replace usage of sys_netdev_info with read_sys_netJoshua Harlow
I've seen cases of unable to read from files as well as the existing os errors so catch io error and skip by using the smarter read_sys_net instead. LP: #1625766
2016-12-02fix problems found in python2.6 test.Joshua Harlow
These are just simple syntax fixes to work correctly on python2.6. Found when testing in a centos 6 container.
2016-11-28OpenStack: extend physical types to include hyperv, hw_veb, vhost_user.Scott Moser
This extends the list of device 'types' that are considered to be physical to include hyperv, hw_veb, and vhost_user. LP: #1642679
2016-11-22tests: fix assumptions that expected no eth0 in system.Scott Moser
The previous commit added tests that would fail on any system that had a nic named eth0 or eno1. The changes here supply the expected macs to the function being tested so it does not query the system. LP: #1644043
2016-11-22net/cmdline: Consider ip= or ip6= on command line not only ip=Scott Moser
The previous behavior would miss ip6= on the command line and would not pay attention to the written net-* or net6-* files if only ip6= was found. The fix here enables parsing the files if either ip= or ip6= is found, and adds some tests as well. LP: #1639930
2016-11-22Just use file logging by defaultJoshua Harlow
Instead of being dependent on the availability of syslog that various distributions may not enable or configure correctly or they do so via patches just use a known-to-work default logging mechanism. If distros want to change this, that is fine, but at least the built-in one will work reliably. LP: #1643990
2016-11-22Improve formatting for ProcessExecutionErrorWesley Wiedenmeier
This replaces long single lines in a log or console output with multiple lines that are much easier to read. It indents the stdout and stderr so logs are more easily read also.
2016-11-22flake8: fix trailing white spaceScott Moser
2016-11-22Doc: various documentation fixesSean Bright
Several various minor fixes for the readthedocs documentation.
2016-11-22cloudinit/config/cc_rh_subscription.py: Remove repos before addingBrent Baude
A user has pointed out that upon set up of a machine, users typically remove repos (sometimes all of them) and then add repos in. This does make sense for a typical user.
2016-11-22packages/redhat: fix rpm spec file.Scott Moser
Adjust the redhat spec file to fix errors found during a ./tools/brpm on centos 6: RPM build errors: File listed twice: /usr/libexec/cloud-init/uncloud-init File listed twice: /usr/libexec/cloud-init/write-ssh-key-fingerprints Installed (but unpackaged) file(s) found: /etc/NetworkManager/dispatcher.d/hook-network-manager /etc/dhcp/dhclient-exit-hooks.d/hook-dhclient
2016-11-22main: set TZ in environment if not already set.Ryan Harper
If TZ is not set in environment, then datetime.datetime() will stat /etc/localtime on every usage. datetime is used in the logging path of cloud-init, and as such this is very busy. During a normal cloud-init boot, it results in we roughly 400+ stat calls to /etc/localtime. The value we set is :/etc/localtime, which is described at https://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html See also merge proposal at 307722 for more background. https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/307722
2016-11-22Azure: No longer rely on walinux agent.Scott Moser
Cloud-init has for some time relied on walinuxagent to do some bits of work necessary for instance initialization. That reliance has not been needed for a while, but we have still defaulted to it. This change uses the "builtin" path that Daniel Watkins added some time ago by default. Also, Adjust tests that assumed the non-__builtin__ Azure agent_command. LP: #1538522
2016-11-22disk_setup: Use sectors as unit when formatting MBR disks with sfdisk.Daniel Watkins
The version of sfdisk in wily (and onwards) only accepts sectors as a valid disk size. As such, this refactors the MBR code path in cc_disk_setup to use sectors. - use --unit=S: while newer versions of sfdisk assume --unit=S, older versions do not so we specifically pass it in. Versions of sfdisk found in supported OSes such as centos6 wont assume --unit=S. - add --force: this exists back to centos 6 (2.17.2), so it should be fine, and is what we ultimately want. "do what I say, even if it is stupid" - keep --Linux. Even though this has been deprecated for quite some time, we keep it until versions that want it are unsupported. If necessary at some point we could check for util linux version and if it had --Linux and use it in those cases. Additionally, improve usefulness of some log messages. LP: #1460715
2016-11-18Add activate_datasource, for datasource specific code paths.Scott Moser
This adds a call to 'activate_datasource'. That will be called during init stage (or init-local in the event of a 'local' dsmode). It is present so that the datasource can do platform specific operations that may be necessary. It is passed the fully rendered cloud-config and whether or not the instance is a new instance. The Azure datasource uses this to address formatting of the ephemeral devices. It does so by a.) waiting for the device to come online b.) removing the marker files for the disk_setup and mounts modules if it finds that the ephemeral device has been reset. LP: #1611074
2016-11-15systemd: cloud-init-local use RequiresMountsFor=/var/lib/cloudScott Moser
While cloud-init writes its data to /var/lib/cloud, we previously lazily added RequiresMountsFor=/var/lib . It is more correct to list the more complete path. LP: #1642062
2016-11-15systemd: cloud-init remove After=systemd-networkd-wait-onlineScott Moser
cloud-init.service cannot currently run After=systemd-networkd-wait-online.service as systemd-networkd.service is After dbus.service, but cloud-init.service is: Before=sysinit.target and sysinit.target is before dbus.service. The result is that we are temporarily preferring for cases where there is systemd-networkd in place to have cloud-init.service run without networking properly configured. See bug 1636912 for more information.
2016-11-15systemd: cloud-init-local change Before basic to sysinitScott Moser
sysinit.target happens earlier in boot than basic.target. cloud-init-local.service had: Before=basic.target but cloud-init.service had: After=cloud-init-local.service Before=sysinit.target The result was that cloud-init-local.service was indirectly Before sysinit.target anyway, so this change is just to more specifically state that.
2016-11-10pep8: fix style errors reported by pycodestyle 2.1.0Scott Moser
pycodestyle 2.1.0 is in Ubuntu zesty, and complained about the changes made here. Simple style changes. This makes 'make pep8' pass again when built in a zesty build system with proposed enabled.
2016-11-10systemd: drop both Wants and After local-fs.targetScott Moser
Drop mention of local-fs.target from both cloud-init-local.service and cloud-init.service. This could actually could cause a loop in ordering due to cc_mounts writing mount points with: x-systemd.requires=cloud-init.service Rather, we specifically list /var/lib/ and systemd-remount-fs.service in cloud-init-local.service. cloud-init.service will run after cloud-init-local and thus will have these satisfied.
2016-11-10systemd: networking service adjustments.Scott Moser
Here we drop the Requires=networking.service but keep the After=networking.service. The change there is that we no longer force networking.service to run, as the system may not have it or be configured to use it. Second, we add After=systemd-networkd-wait-online.service. That causes us to run after systemd-networkd has brought networking up. The end result is that we now run after both ifupdown or systemd-networkd (if they were going to run) but do not force either to run. One other change is that by dropping Requires=networking.service cloud-init will now run even if networking failed to come up. LP: #1636912
2016-11-10systemd: replace Before=basic.target, dbus.target with sysinit.targetScott Moser
Per discussion on bug 1636912 and subsequent feedback on bug 1629797, using: Before=sysinit.target is a simpler solution than what we had implemented: Before=basic.target Before=dbus.target LP: #1629797
2016-11-10doc: Add documentation on stages of boot.Scott Moser
This adds long overdue documentation on stages that cloud-init runs during boot.
2016-11-10doc: make the RST files consistently formated and other improvements.Scott Moser
The biggest things here are: * move doc/sources/*/README.rst to doc/rtd/topics/datasources This gives each datasource a page in the rtd docs, which make it easier to read. * consistently use the same header style throughout. As suggested at http://thomas-cokelaer.info/tutorials/sphinx/rest_syntax.html use: # with overline, for parts * with overline, for chapters =, for sections -, for subsections ^, for subsubsections “, for paragraphs Also, move and re-format vendor-data documentation to rtd.
2016-11-09Ec2: fix syntax and tox in previous commit.Scott Moser
Simply fix a commit that should not have been pushed.
2016-11-08Ec2: protect against non-dictionary in block-device-mapping.Scott Moser
Oracle public cloud has the string 'unavailable' in its metadata service for 'block-device-mapping'. The change here is to return None in device_name_to_device if that is the case.
2016-11-08doc: fixed example to not overwrite /etc/hostsChris Glass
Instead, it will simply append the new entry.
2016-11-07Doc: fix spelling / typos in ca_certs and scripts_vendor.Scott Moser
Simple typo fixes.
2016-11-07pyflakes: fix issue with pyflakes 1.3 found in ubuntu zesty-proposed.Scott Moser
An obvious fix for an issue raised by pyflakes 1.3.
2016-11-03net/cmdline: Further adjustments to ipv6 supportLaMont Jones
The implementation to add ipv6 support to Ubuntu initramfs changed (see bug 1621507). The changes here adjust to handle the new path. Now, the ipv6 route includes using the variable 'DEVICE6' in net6-DEVICE.conf files. LP: #1621615
2016-11-03Add coverage dependency to bddeb to fix package build.Scott Moser
When we added coverage to test-requirements, we need to add the mapping to package name. Without it there, bddeb complains that it cannot translate the dependency. Note, though, that the Makefile does not invoke nose with coverage. So we don't actually use that dependency.
2016-11-01doc: improve HACKING.rst fileScott Moser
Put a bit more information and a few style fixes in HACKING.rst.
2016-11-01dmidecode: Allow dmidecode to be used on aarch64Robert Schweikert
aarch64 systems have functional dmidecode, so allow that to be used. - aarch64 has support for dmidecode as well
2016-11-01AliYun: Add new datasource for Ali-Cloud ECSkaihuan.pkh
Support AliYun(Ali-Cloud ECS). This datasource inherits from EC2, the main difference is the meta-server address is changed to 100.100.100.200. The datasource behaves similarly to EC2 and relies on network polling. As such, it is not enabled by default.
2016-10-28Add coverage collection to tox unit tests.Joshua Powers
First step in increasing coverage is knowing what coverage is currently at. By default, tox only runs coverage on py3 as it is slower to run with coverage.
2016-10-25cc_users_groups: fix remaing call to ds.normalize_user_groupsRyan Harper
Previous commit f0747c4b4cf073273e11d383f0354257be7276ed relocated normalize_users_groups to ug_util module, but missed modifying the cc_users_groups config module. Apply the location change to fix.
2016-10-25disk-config: udev settle after partitioning in gpt format.Scott Moser
The function exec_mkpart_gpt was simply not waiting for udev events to flush after calling sgdisk. The corresponding function exec_mkpart_mbr already did. This should fix a transient failure where mkfs would fail with 'not a block device'. LP: #1626243
2016-10-24unittests: do not read system /etc/cloud/cloud.cfg.dScott Moser
Many of the unit tests in test_data would inadvertantly read the system's /etc/cloud/cloud.cfg and /etc/cloud/cloud.cfg.d. This was first noticed on a system deployed by MAAS, where files in /etc/cloud/cloud.cfg.d/ are root read-only. This changes those tests to actually make use of FilesystemMockingTestCase functionality and adds 'reRoot()' to that class which is easier to use for at least this use case. LP: #1635350