summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2017-05-17Azure: fix reformatting of ephemeral disks on resize to large types.Scott Moser
Large instance types have a different disk format on the newly partitioned ephemeral drive. So we have to adjust the logic in the Azure datasource to recognize that a disk with 2 partitions and an empty ntfs filesystem on the second one is acceptable. This also adjusts the datasources's builtin fs_setup config to remove the 'replace_fs' entry. This entry was previously ignored, and confusing. I've clarified the doc on that also. LP: #1686514
2017-05-16Revert "tools/net-convert: fix argument order for render_network_state"Scott Moser
This reverts commit 64a3df16d9c63db470a3ba55d9c5cc8e05d050d7. Patch shouldn't have been pulled. The issue was inside the netplan renderer.
2017-05-16make deb: Add devscripts dependency for make deb. Cleanup packages/bddeb.Chad Smith
Add a simple dependency check to "make deb" target for devscripts. Rework a bit of the logic in package/bddeb to drop superfluous STD_NAMED_PACKAGES to avoid duplication of requirements already listed in (test-)?requiremets.txt. All "standard" packages can be assumed to have either python3- or python- prefix if not listed in NONSTD_NAMED_PACKAGES. This branch also moves logic inside write_debian_folder which is unneeded up in main. LP: #1685935
2017-05-16tools/net-convert: fix argument order for render_network_stateRyan Harper
We're calling Renderer.render_network_state() with incorrect args. % PYTHONPATH=`pwd` ./tools/net-convert.py \ --network-data=simple-v2.yaml --kind=yaml \ --output-kind netplan --directory ./target Traceback (most recent call last): File "./tools/net-convert.py", line 82, in <module> main() File "./tools/net-convert.py", line 78, in main r.render_network_state(ns, target=args.directory) TypeError: render_network_state() got multiple values for argument 'target' The method signature requires passing <target dir>, <network_state>. This patch fixes the call order. LP: #1685944
2017-05-16openstack: fix log message copy/paste typo in _get_url_settingsLars Kellogg-Stedman
There was a copy/paste error in _get_url_settings such that the error message would complain about max wait when in fact it was talking about retries.
2017-05-16unittests: fix unittests run on centosJoshua Powers
Apt related tests were broken when running on centos becasue apt is not available. This fixes the unit test, with a small re-work of apt_configure. Also in 'tox -e centos6' only run nose on tests/unittests as tests/ also contain integration tests that should not be run.
2017-05-16Improve detection of snappy to include os-release and kernel cmdline.Scott Moser
Recent core snap images (edge channel revision 1886) do not contain the previously known files used to detect that a system is ubuntu core. The changes here are to look in 2 additional locations to determine if a system is snappy. LP: #1689944
2017-05-16Add address to config entry generated by _klibc_to_config_entry.Julien Castets
If /run/net-<name>.cfg contains an IPV4ADDR or an IPV6ADDR, the config file generated by _klibc_to_config_entry now contains the "address". LP: #1691135
2017-05-10sysconfig: Raise ValueError when multiple default gateways are present.Chad Smith
Fixed setting Route.has_set_default_ipv6 or *_ipv4 to track whether a route already has a default gateway defined. The code was setting Route.has_set_default which wasn't checked when raising "duplicate gateway" ValueErrors. Added unit tests to exercise this expected raised ValueError. Also moved is_ipv6 = subnet.get('ipv6') logic out of a for loop because we don't need to recalculate the same value every route iteration. LP: #1687485
2017-05-10FreeBSD: improvements and fixes for use on AzureHongjiang Zhang
This patch targets to make FreeBSD 10.3 or 11 work on Azure. The modifications abide by the rule of: * making as less modification as possible * delegate to the distro or datasource where possible. The main modifications are: 1. network configuration improvements, and movement into distro path. 2. Fix setting of password. Password setting through "pw" can only work through pipe. 3. Add 'root:wheel' to syslog_fix_perms field. 4. Support resizing default file system (ufs) 5. copy cloud.cfg for freebsd to /etc/cloud/cloud.cfg rather than /usr/local/etc/cloud/cloud.cfg. 6. Azure specific changes: a. When reading the azure endpoint, search in a different path and read a different option name (option-245 vs. unknown-245). so, the lease file path should be generated according to platform. b. adjust the handling of ephemeral mounts for ufs filesystem and for finding the ephemeral device. c. fix mounting of cdrom LP: #1636345
2017-05-10Add unit tests for ds-identify, fix Ec2 bug found.Scott Moser
This adds several unit tests for ds-identify, and fixes a bug in Ec2 detection that I found while writing these tests. The method of testing is to use the ds-identify code as a shell library. The TestDsIdentify:call basically does: * populate a (temp) directory with files that represent what ds-identify would see in /sys or other locations it reads. * create a file '_shwrap' that replaces the 3 programs that are executed in ds-identify code path. It supports setting their stdout, stderr, and exit code. * set the default policies explicitly (DI_DEFAULT_POLICY) so we can support testing different builtins. This is necessary because the Ubuntu branches patch the builtin value. If we did not explicilty set it, then testing there would fail. * execute sh to source the script and call its main.
2017-05-03fs_setup: if cmd is specified, use shell interpretation.Paul Meyer
If 'cmd' is provided to a fs_setup entry, then cloud-init was trying to execute the rendered string as a single name, rather than splitting the string. The change here will pass the string to shell for interpretation so that it is split there. Also fix some documentation errors and warn when fs_opts or overwrite is provided along with 'cmd'. LP: #1687712
2017-05-03doc: document network configuration defaults policy and formats.Ryan Harper
Add documentation for cloud-init networking configuration formats, default behavior, policy and other specific details about how network config is consumed and utilized.
2017-04-29Fix name of "uri" key in docs for "cc_apt_configure" moduleFelix Dreissig
The key is called "uri", not "url". This is what's used throughout the examples and also what works in practice (verified on Ubuntu 16.10). This also slightly improves formatting of the key names in the related documentation.
2017-04-26tests: Enable artfulJoshua Powers
2017-04-24nova-lxd: read product_name from environment, not platform.Scott Moser
Apparently signals were crossed when this implementation was done. Cloud-init was reading 'platform' in the environment of pid 1, but nova-lxd was setting 'product_name'. The fix is being made here in cloud-init to instead read product_name. LP: #1685810
2017-04-21Fix yum repo config where keys contain array valuesDylan Perry
ConfigObj produces configuration files that are incompatible with yum if multiple values are listed for a configuration key. Switch to the builtin configparser, and ConfigParser (Python 2) which correctly handles this case. Add additional test case for array values in yum_repos definition LP: #1592150
2017-04-21template: Update debian backports templateJoshua Powers
Debian backports are contains in the main repo and not somewhere seperate. Thanks to Charles Plessy. LP: #1627293
2017-04-21rsyslog: replace ~ with stopJoshua Powers
The discard action (tilde character) has been replaced by the “stop” RainerScript directive. It is considered more intuitive and offers slightly better performance. The tilde operator was deprecated in rsyslog 7. Distributions using rsyslog older than that will need to patch. LP: #1367899
2017-04-21Doc: add additional RTD examplesJoshua Powers
Includes missing examples for RTD, including examples for datasources, disk partitions and apt update. Also fix doc in cloud-config-update-apt.txt. LP: #1459604
2017-04-21Fix growpart for some cases when booted with root=PARTUUID.Scott Moser
Growing the root partition would fail in either of two cases: a.) if the device /dev/root existed b.) the kernel command line had upper case letters in PARTUUID=<value> the kernel will accept upper case partuuid, but udev creates links with lower case. In that scenario, we need to adjust to a /dev/disk/by-<partuuid|uuid> with lower case. The fix here addresses that, and also fixes uuid similarly for the lowercase issue. LP: #1684869
2017-04-21pylint: update output style to parseableJoshua Powers
This makes the output much easier to view and parse through while fixing issues.
2017-04-21pylint: fix all logging warningsJoshua Powers
This will change all instances of LOG.warn to LOG.warning as warn is now a deprecated method. It will also make sure any logging uses lazy logging by passing string format arguments as function parameters.
2017-04-21CloudStack: Add NetworkManager to list of supported DHCP lease dirs.Syed
To query the metadata, the Cloudstack source currently scans a predefined DHCP lease directories to find the IP of the DHCP server. This list does not include "/var/lib/NetworkManager/" which is the default directory in CentOS7. Add that directory to the list.
2017-04-20net: kernel lies about vlans not stealing mac addresses, when they doDimitri John Ledkov
Introduce is_vlan function and call that when building dictionary of interfaces by mac address. LP: #1682871
2017-04-14ds-identify: Check correct path for "latest" config driveDaniel Watkins
We were checking /var/lib/cloud/openstack/latest/meta_data.json instead of /var/lib/cloud/seed/config_drive/openstack/latest/meta_data.json. LP: #1673637
2017-04-12doc: Fix example for resolve.conf configuration.Jon Grimm
Should be 'manage_resolv_conf' not 'manage-resolv-conf'. LP: #1531582
2017-04-12Fix examples that reference upstream chef repository.Jon Grimm
Also add integration test. Note: this new test is not comprehensive; it simply ensures that the example chef configuration does not blow up and that chef seems to be installed after its completion. This new test is disabled by default as it depends on a 3rd party repository. LP: #1678145
2017-04-12doc: correct grammar and improve clarity in merging documentation.David Tagatac
Move merging.rst into doc/rtd/topics with small fixes.
2017-04-12doc: Add missing doc link to snap-config module.Ryan Harper
2017-04-12snap: allows for creating cloud-init snapJoshua Powers
Add a basic snapcraft.yaml file to allow the execution of cloud-init as a snap. This will always pull down the latest source from master for the snap. setup.py will now also set the default init system to be systemd when no other is passed to it.
2017-04-12DigitalOcean: assign IPv4ll address to lowest indexed interface.Ben Howard
Previously the IPv4LL address for metadata discovery was assigned to the first interfaces from an alphabetic sort. On DigitalOcean, the metadata is only accessible from the first interface. This fixes a problem where the IPv4LL address is bound to the wrong interface with snapshots. This is part of general improvements to the DigitalOcean Datasource in bug 1676908.
2017-04-12DigitalOcean: configure all NICs presented in meta-data.Ben Howard
Instead of only configuring 'public' and 'private' interfaces, we want to configure any that has been defined in the meta-data. For legacy reasons, the 'public' and 'private' interfaces are maintained as 'eth0' and 'eth1' respectively. This is part of bug 1676908 for general DigitalOcean datasource fixups.
2017-04-12Remove (and/or fix) URL shortener referencesJon Grimm
Several references that were using URL shorteners are now broken due to their service going away, making it painful to even figure out what they were supposed to be pointing at. Put back long URLS using '# noqa' to make flake8 happy. LP: #1669727
2017-04-12HACKING.rst: more info on filling out contributors agreement.Scott Moser
When signing the Contributors agreement, the user is prompted to provide a 'Project contact' or 'Canonical Project Manager'. Just update the HACKING.rst document to tell them what to put there.
2017-04-12util: teach write_file about copy_mode optionLars Kellogg-Stedman
On centos/fedora/rhel/derivatives, /etc/ssh/sshd_config has mode 0600, but cloud-init unilaterally sets file modes to 0644 when no explicit mode is passed to util.write_file. On ubuntu/debian, this file has mode 0644. With this patch, write_file learns about the copy_mode option, which will cause it to use the mode of the existing file by default, falling back to the explicit mode parameter if the file does not exist. LP: #1644064 Resolves: rhbz#1295984
2017-04-11DigitalOcean: bind resolvers to loopback interface.Ben Howard
This change makes the DigitalOcean datasource consistent with OpenStack and Joyent by binding the resolver addresses to the loopback interface. This _is_ a work-around to bug 1675571. Part of bug 1676908.
2017-04-11tests: fix AltCloud tests to not rely on blkidScott Moser
This just mocks out the AltCloud tests to not invoke blkid. Our tests should not rely on system command returning any specific value. Also, shorten long lines with change in the import name of DataSourceAltCloud. LP: #1636531
2017-04-03OpenStack: add 'dvs' to the list of physical link types.Scott Moser
Links presented in network_data.json to the guest running on ESXi are of type 'dvs'. LP: #1674946
2017-03-31Fix bug that resulted in an attempt to rename bonds or vlans.Scott Moser
When cloud-init ran in the init stage (after networking had come up). A bug could occur where cloud-init would attempt and fail to rename network devices that had "inherited" mac addresses. The intent of apply_network_config_names was always to rename only the devices that were "physical" per the network config. (This would include veth devices in a container). The bug was in creating the dictionary of interfaces by mac address. If there were multiple interfaces with the same mac address then renames could fail. This situation was guaranteed to occur with bonds or vlans or other devices that inherit their mac. The solution is to change get_interfaces_by_mac to skip interfaces that have an inherited mac. Also drop the 'devs' argument to get_interfaces_by_mac. It was non-obvious what the result should be if a device in the input list was filtered out. ie should the following have an entry for bond0 or not. get_interfaces_by_mac(devs=['bond0']) LP: #1669860
2017-03-31tests: update OpenNebula and Digital Ocean to not rely on host interfaces.Scott Moser
Mock the use use of get_interfaces_by_mac in Digital Ocean and OpenNebula. Its best to mock this for the tests as the results aren't expecting it to fail. Note, as it stands, OpenNebula relies on devices named 'eth0'. The metadata (context) does not provide mac addresses.
2017-03-30net: in netplan renderer delete known image-builtin content.Scott Moser
When rendering network configuration to netplan, remove known "builtin" configurations. The specific example here is Ubuntu Core that has netplan configuration in etc/netplan/00-snapd-config.yaml. We also delete the derived files since netplan will have created these derived files in its generator that runs well before cloud-init. LP: #1675576
2017-03-30doc: correct grammar in capabilities.rstDavid Tagatac
2017-03-30ds-identify: fix detecting of maas datasource.Scott Moser
The reading of MAAS datasource configuration was simply broken. it was looking in /etc/cloud/*maas*.cfg rather than /etc/cloud/cloud.cfg.d/*maas*.cfg. along side here there is also: * doc improvement on check_config * remove the path restrictions when searching for values in both maas and ovf_vmware_guest_customization. that was done to improve performance as check_config's parsing is slow. * change to maas to search all config files rather than restricting to a subset as it tried before. that was done for * better variable names. - rename path_cloud_confd to path_etc_cloud - PATH_ETC_CLOUD: /etc/cloud - PATH_ETC_CI_CFG: /etc/cloud/cloud.cfg - PATH_ETC_CI_CFG_D: /etc/cloud/cloud.cfg.d LP: #1677710
2017-03-30netplan: remove debugging prints, add debug loggingRyan Harper
Remove debugging print statements. Change a few to use logging.debug() where useful.
2017-03-29ds-identify: do not write None twice to datasource_list.Scott Moser
If the only the None datasource was listed in datasource_list, then ds-identify would write a cloud.cfg witih: datasource_list: [None, None] The fix is to just append None if the list only has None.
2017-03-29support resizing partition and rootfs on system booted without initramfs.Steve Langasek
When booted without an initramfs, the root device will be /dev/root, not a named device. There is partial support for this when resizing filesystems, but not for growing partitions, without which it doesn't do much good. Move the /dev/root resolution code to util.py and use it from cc_growpart.py. Also, booting without an initramfs only works with a root= argument that's either a kernel device name (which is unstable) or a partition UUID. Handle the case of root=PARTUUID=value, not just LABEL and UUID. LP: #1677376
2017-03-29apt_configure: run only when needed.Scott Moser
Do not bother configuring apt if no 'apt' config is provided and either: a.) running on snappy b.) there is no 'apt' command (possibly a different distro) If apt config is provided in either of the above situations, then config will continue. LP: #1675185
2017-03-28OpenStack: identify OpenStack by product 'OpenStack Compute'.Scott Moser
OpenStack clouds installed with RedHat RDO have the nova product configured in /etc/nova/release to be 'OpenStack Compute' rather than upstream nova default of 'OpenStack Nova'. This was first reported on Finnish provider Nebula (http://nebula.fi). LP: #1675349
2017-03-27GCE: Search GCE in ds-identify, consider serial number in check.Scott Moser
While documentation indicates that the smbios product name should contain 'Google Compute Engine', experimentation and bug reports indicate that is not always the case. The change here is to change the check for GCE to also consider a serial number that starts with 'GoogleCompute-'. Also, ds-identify was not currently searching for GCE if no config of datasource_list was found. Most images have a datasource_list defined. So update the list to include GCE. LP: #1674861