| Age | Commit message (Collapse) | Author |
|---|
|
runcmd, bootcmd, snap/commands, ubuntu-advantage/commands would
log warning (and fail if strict) on duplicate values in the commands.
But those should be allowed. Example, it is perfectly valid to do:
runcmd: ['sleep 1', 'sleep 1']
LP: #1764264
|
|
The net-tools package is deprecated and will eventually be dropped. Use
"ip route", "link" or "address" instead of "ifconfig" or "route" calls.
Cloud-init can now run in an environment that no longer has net-tools.
This affects the network and route printing emitted to
cloud-config-output.log as well as the cc_disable_ec2_metadata module.
Additional changes:
- separate readResource and resourceLocation into standalone test
functions
- Fix ipv4 address rows to report scopes represented by ip addr show
- Formatted route/address ouput now handles multiple ipv4 and ipv6
addresses on a single interface
Co-authored-by: James Hogarth <james.hogarth@gmail.com>
Co-authored-by: Robert Schweikert <rjschwei@suse.com>
|
|
If the metadata service in the host is down while a guest that uses
DataSourceSmartOS is booting, the request from the guest falls into the
bit bucket. When the metadata service is eventually started, the guest
has no awareness of this and does not resend the request. This results in
cloud-init hanging forever with a guest reboot as the only recovery
option.
This fix updates the metadata protocol to implement the initialization
phase, just as is implemented by mdata-get and related utilities. The
initialization phase includes draining all pending data from the serial
port, writing an empty command and getting an expected error message in
reply. If the initialization phase times out, it is retried every five
seconds. Each timeout results in a warning message: "Timeout while
initializing metadata client. Is the host metadata service running?" By
default, warning messages are logged to the console, thus the reason for a
hung boot is readily apparent.
LP: #1667735
|
|
Python has deprecated these invalid string literals now
https://bugs.python.org/issue27364
and pycodestyle is identifying them with a W605 warning.
https://github.com/PyCQA/pycodestyle/pull/676
So basically, any use of \ not followed by one of [\'"abfnrtv]
or \ooo (octal) \xhh (hex) or a newline is invalid. This is most
comomnly seen for us in regex. To solve, you either:
a.) use a raw string r'...'
b.) correctly escape the \ that was not intended to be interpreted.
|
|
If a file passed to render_from_file had non-ascii text then
jinja in python2 would decode as ascii, which would cause
UnicodeDecodeError. This issue can be re-created in python2
with just:
'can\xe2\x80\x99t'.decode()
The solution here is to explicitly pass in unicode supporting
type (py3 str, py2 unicode). Those are six.text_type.
Then jinja does not try to decode.
The reason we hit this is that load_file calls decode_binary.
decode_binary believes it has no work to do if it got a six.string_types.
isinstance('can\xe2\x80\x99t', six.string_types) == True
So it returns the original string which will blow up for jinja.
Our fix here then is to load the file in binary mode and explicitly
decode it to utf-8. Then in python2 we'll have a unicode type
and in python3 we'll have a string type.
|
|
Add a base NTP client configuration dictionary and allow Distro
specific changes to be merged. Add a select client function which
implements logic to preferr installed clients over clients which
need to be installed. Also allow distributions to override the
cloud-init defaults.
LP: #1749722
|
|
Fix integraiton test logic for ec2 to look for network and
availability-zone data under the key path
'ds'=>'meta-data' instead of just 'ds' when parsing instance-data.json.
|
|
Integration tests previously had a logic path that was unexercised on
jenkins because we were on an older version of lxc. With an upgrade to lxd
version 3.0 we need to bump pylxd dependency pin and fix a typo in
integration tests which checked the lxd version.
|
|
LP: #1420018
|
|
The zfs/zpool commands will hang for 10 seconds if /dev/zfs is not
present (bug 1760173). This is a common occurence for containers
using zfs as rootfs. Additionally handle missing zpool command or
other errors that may occur while executing the zpool command.
|
|
This takes the same basic check that is in ds-identify. If the
DMI system manufacturer (aka sys_vendor) is not 'Hetzner', then exit
out of the datasource's get_data quickly.
|
|
Previously there was no support at all for zfs file system. With this
change it is now possible to use the resizefs module to grow a zpool to
its maximum partition size on FreeBSD.
LP: #1721243
|
|
This just got missed in the IBMCloud datasource addition.
Add it to the builtin list of datasources.
|
|
This adds a specific IBM Cloud datasource.
IBM Cloud is identified by:
a.) running on xen
b.) one of a LABEL=METADATA disk or a LABEL=config-2 disk with
UUID=9796-932E
The datasource contains its own config-drive reader that reads
only the currently supported portion of config-drive needed for
ibm cloud.
During the provisioning boot, cloud-init is disabled.
See the docstring in DataSourceIBMCloud.py for more more information.
|
|
Reducing timeout to 1 second as IMDS responds within a handful
of milliseconds. Also get rid of max_retries to prevent exiting
out of polling loop early due to IMDS outage / upgrade.
Reduce Azure PreProvisioning HTTP timeouts during polling to
avoid waiting an extra minute.
LP: #1752977
|
|
ubuntu-advantage-tools is a package for enabling and disabling extended
support services such as Extended Security Maintenance (ESM), Canonical
Livepatch and FIPS certified PPAs. Simplify Ubuntu Advantage setup on
machines by allowing users to provide a list of ubuntu-advantage commands
in cloud-config.
|
|
In network config v1 format, there are dns values which are not bound to a
specific interface and do not map to the per-interface format in netplan.
To handle this case we render netplan configuration that duplicates the
DNS configuration on any interface that has a static network config. We
avoiding interfaces which have DHCP configuration which may provide
conflicting DNS values.
LP: #1750884
|
|
Open Telekom Cloud gen1 (Xen) hosts do not provide nova product
names in DMI but Xen HVM domU. They can however be safely identified
by the OpenTelekomCloud Chassis asset tag. OpenTelekomCloud does
use the network OpenStack DataSource, so we better detect it.
LP: #1756471
|
|
OpenNebulaNetwork.gen_conf() was previously returning ENI format.
This is updated to return netplan/v2 config.
The changes here also adds support for IPv6 configuration distributed
from OpenNebula and fixes some issues about nameserver information.
|
|
The Hetzner Cloud metadata service is an AWS-style service available
over HTTP via the link local address 169.254.169.254.
https://hetzner.com/cloud
https://docs.hetzner.cloud/
|
|
When 'ip=' or 'ip6=' is found on the kernel command line,
cloud-init will consider read network config from /run/net-*.conf files.
There are some iscsi-root scenarios where initramfs configures networking
but the ip= parameter is not present. 2 such cases are:
a.) static config in /etc/iscsi/iscsi.initramfs (copied into the
initramfs)
b.) iBft
This changes cloud-init to consider initramfs provided networking
information if:
* there are /run/net-* files and
* (ip= or ip6 is on the command line) or open-iscsi.interface file
exists.
LP: #1752391
|
|
Support installing and configuring snaps on ubuntu systems. Now,
cloud-config files can provide a list or dictionary of snap:assertions
which will be allow configuration of snapd on a system via 'snap ack'
calls. The snap:commands configuration option supports arbitrary system
commands intended to interact with snappy's cli. This allows users to run
arbitrary snappy commands to create users, download, install and
configure snap packages and snapd.
This branch also deprecates old snappy and snap_config modules leaving
warnings in documentation and runtime for consumers of these modules.
Deprecated snap* modules will be dropped in cloud-init v.18.2 release.
|
|
Older unittest2.TestCase (as seen in CentOS 6) do not have an
assertRaisesRegex method. They only have the now-deprecated
assertRaisesRegexp.
We need our unit tests to work there and on newer python (3.6).
Simply making assertRaisesRegex = assertRaisesRegexp makes pylint
complain as described in https://github.com/PyCQA/pylint/issues/1946 .
What was here before this commit was actually broken. This commit
makes assertRaisesRegex functional in CentOS 6 and works around
the invalid Deprecated warning from pylint.
To prove this, we use assertRaisesRegex in a unit test which will
be exectued in py27, py3 and py26.
|
|
Update netplan renderer to write out bridge port-priority values
now that netplan supports the feature.
LP: #1735821
|
|
The command provided to subp can either be a string or a list. This patch
fixes a regression which raised CalledProcessError whenever providing a
string to subp.
LP: #1755965
|
|
This was broken probably when we inserted the ssh keys into Platform.
tox -e citest tree_run
and
tox -e citest bddeb
would fail with KeyError in Platform.init due to lack of a data_dir.
Also here are a few fixes found from attempting to make it work.
|
|
When running 'tox -e pylint' on a bionic system (python 3.6.4) I started
seeing errors today like:
tests/cloud_tests/platforms/__init__.py:5: [E0401(import-error), ]
Unable to import 'tests.cloud_tests.platforms.ec2'
The fix for those first errors was simply to create the __init__.py.
The second set of changes fixes fallout found from actually now having
pylint properly run on more of the cloud_tests.
|
|
When instance meta-data provides hostname information, run
cc_set_hostname in the init-local or init-net stage before network
comes up.
Prevent an initial DHCP request which leaks the stock cloud-image default
hostname before the meta-data provided hostname was processed.
A leaked cloud-image hostname adversely affects Dynamic DNS which
would reallocate 'ubuntu' hostname in DNS to every instance brought up by
cloud-init. These instances would only update DNS to the cloud-init
configured hostname upon DHCP lease renewal.
This branch extends the get_hostname methods in datasource, cloud and
util to limit results to metadata_only to avoid extra cost of querying
the distro for hostname information if metadata does not provide that
information.
LP: #1746455
|
|
This just centralizes a hunk of duplicated code and uses it from the
new location.
|
|
LP: #1754495
|
|
This makes 2 changes to shellify's behavior:
a.) raise a TypeError rather than a RuntimeError.
b.) raise a TypeError if input is not a list or tuple.
|
|
Previously the module was not working under FreeBSD due to a different
package name and some different paths. The module now has OS specific
default values which can even be customized via corresponding cloud config
variables.
LP: #1721503
|
|
Last set of changes to GCE datasource broke reading of user-data
unless the user had base64 encoded their user-data and also set
user-data-encoding to 'base64'.
This fixes the issue.
LP: #1752711
|
|
If python starts up without a locale set, then its default encoding
ends up set as ascii. That is not easily changed with the likes of
setlocale. In order to avoid UnicodeDecodeErrors cloud-init will
encode to bytes a python3 string or python2 basestring so that the
values passed to Popen are already bytes.
LP: #1751051
|
|
While salt grains can be configured in the minion config file, it is
usually better to configure it in the /etc/salt/grains file.
This allows that to be done.
|
|
On few 64-bit platforms, the open-vm-tools package is installed at
/usr/lib64/. The DataSourceOVF is changed to search look there for the
'customization plugin'
|
|
This fixes a bug in parsing of 'blkid -o export' output. The result
of the bug meant that DI_ISO9660_DEVS did not get set correctly and
is_cdrom_ovf would not identify devices in most cases.
The tests are improved to demonstrate both multiple iso devices
and also a cdrom that doesn't sort "last" in blkid output.
The code change is to use DEVNAME as the record separator when
parsing blkid -o export rather than relying on being able to read
the empty line.
LP: #1749980
|
|
Ubuntu core seeds information to nocloud via a bind-mount of
/writable/system-data/var/lib/cloud over /var/lib/cloud.
When ds-identify runs as a systemd generator that mount is not
guaranteed to have been done. It is guaranteed at
cloud-init-local.service time, but not generator time.
Images built with 'ubuntu-image --cloud-init=user-data-file'
would have cloud-init disabled.
The fix here is just to consider the seed dir under /writable/system-data.
LP: #1747070
|
|
When we moved some tests to live under cloudinit/ we inadvertantly
failed to change all things that would run nose to include that
directory.
This changes all the 'nose' invocations to consistently run with
tests/unittests and cloudinit/.
Also, it works around, more correctly this time, a python2.6-ism with
the following code:
with assertRaises(SystemExit) as cm:
sys.exit(2)
|
|
This puts in place detection for if 'show-log' will work with lxc
client, and uses that if present. The 'lxc console --show-log' is
not expected to work until lxd/liblxc3.0. That should come in a
few months. The hope is that when that function arrives, this
code will move over to using it.
For other scenarios (all current lxd installs) this will now
support getting logs from a snap installed lxd or a package installed
lxd via the old 'lxc.console.logfile'.
If installed from snap, a platform error will be raised until
the user does:
sudo mkdir --mode=1777 -p /var/snap/lxd/common/consoles
LP: #1745663
|
|
Fix an issue in EC2 where the datasource.identity had not been
initialized before being used when restoring datasource from pickle.
This is exposed in upgrade and reboot path.
LP: #1748354
|
|
This fixes a bug where invalid keys would sneak into authorized_keys.
|
|
Fujitsu Cloud Service attaches a ovf iso transport with a label
'OVFENV'. This seems to be a reasonable value as a label.
While the for bug 1731868 would likely fix cloud-init on fujitsu
cloud, this change will find it faster.
LP: #1698669
|
|
This adds collection a gzip compressed systemd journal on systemd systems.
The file can later be reviewed with:
zcat system.journal.gz > system.journal
journalctl --file=system.journal [-o short-monotonic ..]
To support this:
* modify test harness infrastructure to not assume content is utf-8.
* fix lxd platform to support make '_execute' return bytes rather
than a string. https://github.com/lxc/pylxd/issues/268
Also switched the base collectors to use /bin/sh as others already did.
|
|
This change will enable azure vms to report provisioning has completed
twice, first to tell the fabric it has completed then a second time to
enable customer settings. The datasource for the second provisioning is
the Instance Metadata Service (IMDS),and the VM will poll indefinitely for
the new ovf-env.xml from IMDS.
This branch introduces EphemeralDHCPv4 which encapsulates common logic
used by both DataSourceEc2 an DataSourceAzure for temporary DHCP
interactions without side-effects.
LP: #1734991
|
|
Resize of btrfs fails if the mount point for the file system we are trying
to resize, i.e. the root of the filesystem is read only. With this change
we use a known (currently snapper specific) rw location to work around a
flaw that blocks resizing of the ro filesystem.
LP: #1734787
|
|
Network configuration in OpenNebula would only work if the host correctly
guessed the names of the devices in the guest. OpenNebula provided data
in its context.sh like 'ETH0_NETWORK', but if the guest named devices
differently then results were not predictable. This would occur with
Predictable Network Interface Names. To address this,
newer versions (of OpenNebula provide the mac address ETH0_MAC.
This function is present in 4.14 and documented officially in 5.0 docs.
This provides support for reading the mac addresses from the context.sh.
It also fixes cases where context.sh provided a field (ETH0_NETWORK
or ETH0_MASK) with a empty string. Previously the empty string would
be used rather than falling back to the default.
LP: #1719157, #1716397, #1736750
|
|
The EC2 test platform uses boto, and boto decodes console output
with decode('utf-8', 'replace). It is known that Ubuntu consoles
contain non-utf8 characters, making this call lossy.
The change here is to patch the boto session to include a OutputBytes
entry in the console_output response, and then to utilize that in
console_log.
More information on problem and solution at:
https://github.com/boto/botocore/issues/1351
|
|
The previous commit added a test that would attempt to create and use
/run/cloud-init/. This just modifies it to use a temp dir instead.
|
|
The behavior changes and improvements include:
- Only import keys into the default user that contain the name of the
default user ('ubuntu', or 'centos') or that contain 'cloudinit'.
- Use instance or project level keys based on GCE convention.
- Respect expiration time when keys are set.
Do not import expired keys.
- Support ssh-keys in project level metadata (the GCE default).
As part of this change, we also update the request header when talking
to the metadata server based on the documentation:
https://cloud.google.com/compute/docs/storing-retrieving-metadata#querying
LP: #1670456, #1707033, #1707037, #1707039
|
