| Age | Commit message (Collapse) | Author |
|---|
|
When attempting to apply network configuration for SmartOS's container
platform, cloud-init would not identify nics. The nics on provided
in this container service do not have 'addr_assign_type'. That
was being interpreted as being a "stolen" mac, and would be filtered
out by get_interfaces.
|
|
package_update_upgrade_install was failing as htop is now included in
Bionic images. Switch this test to install 'sl' instead.
ca_certs integration test fails on cert_count test because bionic
update-ca-certificates on bionic generates less symlinks for a given cert.
Integration tests now collect dpkg-query --show output on every instance.
Add a new assertPackageInstalled helper method which finds the package or
package version installed on the instance.
Adapt existing byobu, package_update_upgrade_install, ntp and salt_minion
tests to use assertPackageInstalled method.
LP: #1769985
|
|
Fix remaining pycodesytle warnings related to invalid string literals
introduced in more recent pycodeflakes versions
https://bugs.python.org/issue27364 .
Also stop using flake8 in tox as it is incompatible with newer versions of
pyflakes. Instead we now add tox environments for pycodestyle and pyflakes
individually.
Set the versions in both pycodestyle and pyflakes to the currently
available versions.
|
|
This change is for Azure VM Preprovisioning. A bug was found when after
azure VMs report ready the first time, during the time when VM is polling
indefinitely for the new ovf-env.xml from Instance Metadata Service
(IMDS), if a reboot happens, we send another report ready signal to the
fabric, which deletes the reprovisioning data on the node.
This marker file is used to fix this issue so that we will only send a
report ready signal to the fabric when no marker file is present. Then,
create a marker file so that when a reboot does occur, we check if a
marker file has been created and decide whether we would like to send the
repot ready signal.
LP: #1765214
|
|
By default, FreeBSD's growfs runs interactively asking a question
which can be mitigated using the '-y' command line option. The fix
here is simply to pass -y to growfs to avoid the prompt.
LP: #1404745
|
|
The last set of changes to netdev_pformat ended up dropping the output
of devices that were not up. This adds back the 'down' interfaces to the
rendered output.
LP: #1766302
|
|
Ubuntu images on IBMCloud for 16.04 have some seed data in
/var/lib/cloud/data/seed/nocloud-net. In order to have systems with
IBMCloud enabled, we modified ds-identify detection to skip that seed
if the system was on IBMCloud. That change did not consider the
fact that IBMCloud might not be in the datasource list.
There was similar logic in the ConfigDrive datasource in ds-identify
and the datasource itself.
Config drive is now updated to only check and avoid IBMCloud if IBMCloud
is enabled. The check in ds-identify for nocloud was dropped. If a
user provides a nocloud seed on IBMCloud, then that can be used.
This means that systems running Xenial will continue to get their
old datasources.
LP: #1766401
|
|
When images are deployed from template in a production environment
the artifacts of the provisioning stage (provisioningConfiguration.cfg)
that cloud-init referenced are cleaned up. However, when provisioned
in "debug" mode (internal to IBM) the artifacts are left.
This changes the 'is_ibm_provisioning' implementations in both
ds-identify and in the IBM datasource to identify the provisioning
stage more correctly. The change is to consider provisioning only
if the provisioing file existed and there was no log file or
the log file was older than this boot.
LP: #1767166
|
|
The cloud-init-local.service expects that any network device name changes
have already been completed by the kernel or udev daemon.
In some situations we've found that the renaming of interfaces from kernel
names (eth0, eth1, etc) to their persistent names (eno1, ens3, enp0s1,
etc) may happen after cloud-init-local has started where it reads values
from sysfs about what network devices are present, and which device to use
as a fallback nic.
Subsequently, cloud-init-local would write out network configuration for a
kernel device name which would no longer be present by the time that
networking services start to bring up the devices. The result is that the
instance does not get networking configured. Prior to use of
systemd-networkd, the Ubuntu 'networking.service' unit included a call to
udevadm settle which is why this race is not seen on a Xenial system.
This change adds the ability to detect if an interface has a stable name,
if if we find one without stable names and stable names have not been
disabled (net.ifnames=0 in /proc/cmdline), then cloud-init will invoke
udevadm settle.
LP: #1766287
|
|
BOOTPROTO=dhcp in sysconfig enables DHCPv4 and we should not do this
implicitly when 'dhcp6' subnet is specified. In case both dhcpv4 and
dhcpv6 are needed users should specify both:
subnets:
- type: dhcp6
- type: dhcp
Fix the current code and add a dhcpv6 only test.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
|
|
cloud-init and mdata-get each have their own implementation of the
SmartOS metadata protocol. If cloud-init and other services that call
mdata-get are run concurrently, crosstalk on the serial port can cause
them both to become confused.
This change makes it so that cloud-init uses the same cooperative
locking scheme that's used by mdata-get, thus preventing cross-talk
between mdata-get and cloud-init.
For testing, a VM running on a SmartOS host and pyserial are required.
If the tests are run on a platform other than SmartOS, those that use a
real serial port are skipped. pyserial remains commented in
requirements.txt because most testers will not be running atop SmartOS.
LP: #1746605
|
|
There are three potential sources of the hostname, one of which is
documented SmartOS's vmadm(1M) via the hostname property. That
property's value is retrieved via the sdc:hostname key. The other
two sources for the hostname are a hostname key in customer_metadata
and the VM's uuid (sdc:uuid). Of these three, the sdc:hostname value
is not used in a meaningful way by DataSourceSmartOS.
This fix changes the fallback mechanism when hostname is not
specified in customer_metadata. The order of precedence for setting
the hostname is now 1) hostname in customer_metadata,
2) sdc:hostname, then 3) sdc:uuid.
LP: #1765085
|
|
If customer_metadata has no keys, the KEYS request returns an empty
string. Callers of the list() method expect a list to be returned and
will give a stack trace if this expectation is not met.
LP: #1763480
|
|
validate_cloudconfig_schema with strict=True would not actually validate
if there was no jsonschema available. That seems kind of strange.
The change here is to make it raise an exception if strict was passed in.
And then to fix the one test that needed a skipIfJsonSchema wrapper.
|
|
This admittedly does a fairly extensive re-factor to simply add a newline
to the end of sshd_config.
It makes the ssh_config updating portion of set_passwords more testable
and adds tests for that.
The new function is in 'update_ssh_config_lines' which allows you
to update a config with multiple changes even though only a single one
is currently used.
We also only restart the ssh daemon now if a change was made to the
config file. Before it was always restarted if the user specified
a value for ssh_pwauth other than 'unchanged'.
Thanks to Lorens Kockum for initial diagnosis and patch.
LP: #1677205
|
|
This enables warnings produced by pylint for unused variables (W0612),
and fixes the existing errors.
|
|
runcmd, bootcmd, snap/commands, ubuntu-advantage/commands would
log warning (and fail if strict) on duplicate values in the commands.
But those should be allowed. Example, it is perfectly valid to do:
runcmd: ['sleep 1', 'sleep 1']
LP: #1764264
|
|
The net-tools package is deprecated and will eventually be dropped. Use
"ip route", "link" or "address" instead of "ifconfig" or "route" calls.
Cloud-init can now run in an environment that no longer has net-tools.
This affects the network and route printing emitted to
cloud-config-output.log as well as the cc_disable_ec2_metadata module.
Additional changes:
- separate readResource and resourceLocation into standalone test
functions
- Fix ipv4 address rows to report scopes represented by ip addr show
- Formatted route/address ouput now handles multiple ipv4 and ipv6
addresses on a single interface
Co-authored-by: James Hogarth <james.hogarth@gmail.com>
Co-authored-by: Robert Schweikert <rjschwei@suse.com>
|
|
If the metadata service in the host is down while a guest that uses
DataSourceSmartOS is booting, the request from the guest falls into the
bit bucket. When the metadata service is eventually started, the guest
has no awareness of this and does not resend the request. This results in
cloud-init hanging forever with a guest reboot as the only recovery
option.
This fix updates the metadata protocol to implement the initialization
phase, just as is implemented by mdata-get and related utilities. The
initialization phase includes draining all pending data from the serial
port, writing an empty command and getting an expected error message in
reply. If the initialization phase times out, it is retried every five
seconds. Each timeout results in a warning message: "Timeout while
initializing metadata client. Is the host metadata service running?" By
default, warning messages are logged to the console, thus the reason for a
hung boot is readily apparent.
LP: #1667735
|
|
Python has deprecated these invalid string literals now
https://bugs.python.org/issue27364
and pycodestyle is identifying them with a W605 warning.
https://github.com/PyCQA/pycodestyle/pull/676
So basically, any use of \ not followed by one of [\'"abfnrtv]
or \ooo (octal) \xhh (hex) or a newline is invalid. This is most
comomnly seen for us in regex. To solve, you either:
a.) use a raw string r'...'
b.) correctly escape the \ that was not intended to be interpreted.
|
|
If a file passed to render_from_file had non-ascii text then
jinja in python2 would decode as ascii, which would cause
UnicodeDecodeError. This issue can be re-created in python2
with just:
'can\xe2\x80\x99t'.decode()
The solution here is to explicitly pass in unicode supporting
type (py3 str, py2 unicode). Those are six.text_type.
Then jinja does not try to decode.
The reason we hit this is that load_file calls decode_binary.
decode_binary believes it has no work to do if it got a six.string_types.
isinstance('can\xe2\x80\x99t', six.string_types) == True
So it returns the original string which will blow up for jinja.
Our fix here then is to load the file in binary mode and explicitly
decode it to utf-8. Then in python2 we'll have a unicode type
and in python3 we'll have a string type.
|
|
Add a base NTP client configuration dictionary and allow Distro
specific changes to be merged. Add a select client function which
implements logic to preferr installed clients over clients which
need to be installed. Also allow distributions to override the
cloud-init defaults.
LP: #1749722
|
|
Fix integraiton test logic for ec2 to look for network and
availability-zone data under the key path
'ds'=>'meta-data' instead of just 'ds' when parsing instance-data.json.
|
|
Integration tests previously had a logic path that was unexercised on
jenkins because we were on an older version of lxc. With an upgrade to lxd
version 3.0 we need to bump pylxd dependency pin and fix a typo in
integration tests which checked the lxd version.
|
|
LP: #1420018
|
|
The zfs/zpool commands will hang for 10 seconds if /dev/zfs is not
present (bug 1760173). This is a common occurence for containers
using zfs as rootfs. Additionally handle missing zpool command or
other errors that may occur while executing the zpool command.
|
|
This takes the same basic check that is in ds-identify. If the
DMI system manufacturer (aka sys_vendor) is not 'Hetzner', then exit
out of the datasource's get_data quickly.
|
|
Previously there was no support at all for zfs file system. With this
change it is now possible to use the resizefs module to grow a zpool to
its maximum partition size on FreeBSD.
LP: #1721243
|
|
This just got missed in the IBMCloud datasource addition.
Add it to the builtin list of datasources.
|
|
This adds a specific IBM Cloud datasource.
IBM Cloud is identified by:
a.) running on xen
b.) one of a LABEL=METADATA disk or a LABEL=config-2 disk with
UUID=9796-932E
The datasource contains its own config-drive reader that reads
only the currently supported portion of config-drive needed for
ibm cloud.
During the provisioning boot, cloud-init is disabled.
See the docstring in DataSourceIBMCloud.py for more more information.
|
|
Reducing timeout to 1 second as IMDS responds within a handful
of milliseconds. Also get rid of max_retries to prevent exiting
out of polling loop early due to IMDS outage / upgrade.
Reduce Azure PreProvisioning HTTP timeouts during polling to
avoid waiting an extra minute.
LP: #1752977
|
|
ubuntu-advantage-tools is a package for enabling and disabling extended
support services such as Extended Security Maintenance (ESM), Canonical
Livepatch and FIPS certified PPAs. Simplify Ubuntu Advantage setup on
machines by allowing users to provide a list of ubuntu-advantage commands
in cloud-config.
|
|
In network config v1 format, there are dns values which are not bound to a
specific interface and do not map to the per-interface format in netplan.
To handle this case we render netplan configuration that duplicates the
DNS configuration on any interface that has a static network config. We
avoiding interfaces which have DHCP configuration which may provide
conflicting DNS values.
LP: #1750884
|
|
Open Telekom Cloud gen1 (Xen) hosts do not provide nova product
names in DMI but Xen HVM domU. They can however be safely identified
by the OpenTelekomCloud Chassis asset tag. OpenTelekomCloud does
use the network OpenStack DataSource, so we better detect it.
LP: #1756471
|
|
OpenNebulaNetwork.gen_conf() was previously returning ENI format.
This is updated to return netplan/v2 config.
The changes here also adds support for IPv6 configuration distributed
from OpenNebula and fixes some issues about nameserver information.
|
|
The Hetzner Cloud metadata service is an AWS-style service available
over HTTP via the link local address 169.254.169.254.
https://hetzner.com/cloud
https://docs.hetzner.cloud/
|
|
When 'ip=' or 'ip6=' is found on the kernel command line,
cloud-init will consider read network config from /run/net-*.conf files.
There are some iscsi-root scenarios where initramfs configures networking
but the ip= parameter is not present. 2 such cases are:
a.) static config in /etc/iscsi/iscsi.initramfs (copied into the
initramfs)
b.) iBft
This changes cloud-init to consider initramfs provided networking
information if:
* there are /run/net-* files and
* (ip= or ip6 is on the command line) or open-iscsi.interface file
exists.
LP: #1752391
|
|
Support installing and configuring snaps on ubuntu systems. Now,
cloud-config files can provide a list or dictionary of snap:assertions
which will be allow configuration of snapd on a system via 'snap ack'
calls. The snap:commands configuration option supports arbitrary system
commands intended to interact with snappy's cli. This allows users to run
arbitrary snappy commands to create users, download, install and
configure snap packages and snapd.
This branch also deprecates old snappy and snap_config modules leaving
warnings in documentation and runtime for consumers of these modules.
Deprecated snap* modules will be dropped in cloud-init v.18.2 release.
|
|
Older unittest2.TestCase (as seen in CentOS 6) do not have an
assertRaisesRegex method. They only have the now-deprecated
assertRaisesRegexp.
We need our unit tests to work there and on newer python (3.6).
Simply making assertRaisesRegex = assertRaisesRegexp makes pylint
complain as described in https://github.com/PyCQA/pylint/issues/1946 .
What was here before this commit was actually broken. This commit
makes assertRaisesRegex functional in CentOS 6 and works around
the invalid Deprecated warning from pylint.
To prove this, we use assertRaisesRegex in a unit test which will
be exectued in py27, py3 and py26.
|
|
Update netplan renderer to write out bridge port-priority values
now that netplan supports the feature.
LP: #1735821
|
|
The command provided to subp can either be a string or a list. This patch
fixes a regression which raised CalledProcessError whenever providing a
string to subp.
LP: #1755965
|
|
This was broken probably when we inserted the ssh keys into Platform.
tox -e citest tree_run
and
tox -e citest bddeb
would fail with KeyError in Platform.init due to lack of a data_dir.
Also here are a few fixes found from attempting to make it work.
|
|
When running 'tox -e pylint' on a bionic system (python 3.6.4) I started
seeing errors today like:
tests/cloud_tests/platforms/__init__.py:5: [E0401(import-error), ]
Unable to import 'tests.cloud_tests.platforms.ec2'
The fix for those first errors was simply to create the __init__.py.
The second set of changes fixes fallout found from actually now having
pylint properly run on more of the cloud_tests.
|
|
When instance meta-data provides hostname information, run
cc_set_hostname in the init-local or init-net stage before network
comes up.
Prevent an initial DHCP request which leaks the stock cloud-image default
hostname before the meta-data provided hostname was processed.
A leaked cloud-image hostname adversely affects Dynamic DNS which
would reallocate 'ubuntu' hostname in DNS to every instance brought up by
cloud-init. These instances would only update DNS to the cloud-init
configured hostname upon DHCP lease renewal.
This branch extends the get_hostname methods in datasource, cloud and
util to limit results to metadata_only to avoid extra cost of querying
the distro for hostname information if metadata does not provide that
information.
LP: #1746455
|
|
This just centralizes a hunk of duplicated code and uses it from the
new location.
|
|
LP: #1754495
|
|
This makes 2 changes to shellify's behavior:
a.) raise a TypeError rather than a RuntimeError.
b.) raise a TypeError if input is not a list or tuple.
|
|
Previously the module was not working under FreeBSD due to a different
package name and some different paths. The module now has OS specific
default values which can even be customized via corresponding cloud config
variables.
LP: #1721503
|
|
Last set of changes to GCE datasource broke reading of user-data
unless the user had base64 encoded their user-data and also set
user-data-encoding to 'base64'.
This fixes the issue.
LP: #1752711
|
|
If python starts up without a locale set, then its default encoding
ends up set as ascii. That is not easily changed with the likes of
setlocale. In order to avoid UnicodeDecodeErrors cloud-init will
encode to bytes a python3 string or python2 basestring so that the
values passed to Popen are already bytes.
LP: #1751051
|
