| Age | Commit message (Collapse) | Author |
|---|
|
blkid is a Linux-only command. With this patch, cloud-init uses another
approach to find the data source on FreeBSD.
LP: #1645824
|
|
The Azure data source helper attempts to use information in the dhcp
lease to find the Wireserver endpoint (IP address). Under some unusual
circumstances, those attempts will fail. This change uses a static
address, known to be always correct in the Azure public and sovereign
clouds, when the helper fails to locate a valid dhcp lease. This
address is not guaranteed to be correct in Azure Stack environments;
it's still best to use the information from the lease whenever possible.
|
|
Bump the version on cloudinit/version.py to be 19.1 and update ChangeLog
LP: #1828479
|
|
Under some circumstances, cc_disk_setup may reformat volumes which
already appear in /etc/fstab (e.g. Azure ephemeral drive is reformatted
from NTFS to ext4 after service-heal). Normally, cc_mounts only calls
mount -a if it altered /etc/fstab. With this change cc_mounts will read
/proc/mounts and verify if configured mounts are already mounted and if
not raise flag to request a mount -a. This handles the case where no
changes to fstab occur but a mount -a is required due to change in
underlying device which prevented the .mount unit from running until
after disk was reformatted.
LP: #1825596
|
|
|
|
If the IMDS primary server is not available, falling back to the
secondary server takes about 1s. The net result is that the
expected E2E time is slightly more than 1s. This change increases
the timeout to 2s to prevent the infinite loop of timeouts.
|
|
+ Truncate KVP Pool file to prevent stale entries from
being processed by the Hyper-V KVP reporter.
+ Drop filtering of KVPs as it is no longer needed.
+ Batch appending of existing KVP entries.
|
|
Currently on 18.04, running tox -e py27 will spew errors like:
.tests/unittests/test_net.py:2649: YAMLLoadWarning: calling yaml.load()
without Loader=... is deprecated, as the default Loader is unsafe.
Please read https://msg.pyyaml.org/load for full details.
The change here just uses cloud-init's yaml, which does safeloading
by default.
|
|
Restrict the sysconfig renderer availabily to known distros.
Ubuntu/Debian systems may include network-manager but they do
not have support for reading sysconfig network output; that is
enabled via a Network-Manager plugin: ifcfg-rh which is not
available in Ubuntu/Debian.
LP: #1819994
|
|
On FreeBSD, mount_cd9660 does not accept the sync option that is enabled
by default. In addition, the sync is only useful with the `rw` mode.
However the `rw` mode was never used.
This patch removes the `rw` and `sync` parameter of `mount_cb` to
simplify the code base and resolve the FreeBSD issue.
LP: #1645824
|
|
|
|
Moving update_events from a class attribute to an instance attribute
means that it doesn't exist on DataSource objects that are unpickled,
causing tracebacks on cloud-init upgrade.
As this change is only required for cloud-init installations which don't
utilise ds-identify, we're backing it out to be reintroduced once the
upgrade path bug has been addressed.
This reverts commit f2fd6eac4407e60d0e98826ab03847dda4cde138.
|
|
NoCloud data source now accepts both 'cidata' and 'CIDATA'
as filesystem labels. This is similar to DataSourceConfigDrive's
support for 'config-2' and 'CONFIG-2'.
|
|
Cloud-init's main.py will fail when presented with a new
stage name 'modules-init' if upgrading an older cloud-init.
Fix this by initializing unknown stage names before accessing.
LP: #1815109
|
|
ubuntu-advantage-tools version 19 has a different command line
interface. Update cloud-init's config module to accept new
ubuntu_advantage configuration settings.
* Underscores better than hyphens: deprecate 'ubuntu-advantage'
cloud-config key in favor of 'ubuntu_advantage'
* Attach machines with either sso credentials of UA user_token
* Services are enabled by name though an 'enable' list
* Raise warnings if deprecated ubuntu-advantage config keys are
present, or errors if its config we cannott adapt to
Ubuntu Advantage support can now be configured via #cloud-config
with the following yaml:
ubuntu_advantage:
token: 'thisismyubuntuadvantagetoken'
enable: [esm, fips, livepatch]
Co-Authored-By: Daniel Watkins <daniel.watkins@canonical.com>
|
|
When the Azure datasource persists all of its metadata to the
instance directory, it deliberately sets the self.network_config
value to be the sources.UNSET value. The goal is to ensure that
each time the system boots, fresh network configuration data is
fetched from the cloud platform so that any control plane changes
will take effect. When a VM is first created, there's no pickled
instance to restore, so self._network_config is None, resulting
in self.network_config() properly building a new config. Azure
suffered from LP: #1801364 which prevented ds from being stored
in obj.pkl in the instance directory, so subsequent reboots always
regenerated their network configuration.
Commit 0dc3a77f41f4544e4cb5a41637af7693410d4cdf introduced a
new bug in which self.network_config() assumed the
self._network_config value was either None or trustable; when
the config was unpickled, that value was _unset, thus breaking
the assumption.
LP: #1823084
|
|
Create an Azure logging decorator and use additional ReportEventStack
context managers to provide additional logging details.
|
|
The Azure platform surfaces random bytes into /sys via Hyper-V.
Python 2.7 json.dump() raises an exception if asked to convert
a str with non-character content, and python 3.0 json.dump()
won't serialize a "bytes" value. As a result, c-i instance
data is often not written by Azure, making reboots slower (c-i
has to repeat work).
The random data is base64-encoded and then decoded into a string
(str or unicode depending on the version of Python in use). The
base64 string has just as many bits of entropy, so we're not
throwing away useful "information", but we can be certain
json.dump() will correctly serialize the bits.
|
|
For writing IPv6 addresses to ifcfg-* the name "IPV6ADDR" is used. For
secondary IPs the value for "IPV6ADDR_SECONDARIES" is set. On SUSE based
distributions the names "IPADDR6" and "IPADDR6_$SOMELABEL" need to be
used.
|
|
When rendering ipv6 static routes in eni format the
post-up/pre down commands were not correct for ipv6.
LP: #1818669
|
|
The ubuntu_drivers config module enables usage of the 'ubuntu-drivers'
command. At this point it only serves as a way of installing NVIDIA
drivers for general purpose graphics processing unit (GPGPU)
functionality.
Also, a small usability improvement to get_cfg_by_path to allow it to
take a string for the key path
"toplevel/second/mykey"
in addition to the original:
("toplevel", "second", "mykey")
|
|
Currently, DataSourceAzure updates self.update_events in __init__. As
update_events is a class attribute on DataSource, this updates it for
all instances of classes derived from DataSource including those for
other clouds. This means that if DataSourceAzure is even instantiated,
its behaviour is applied to whichever data source ends up being used for
boot.
To address this, update_events is moved from a class attribute to an
instance attribute (that is therefore populated at instantiation time).
This retains the defaults for all DataSource sub-class instances, but
avoids them being able to mutate the state in instances of other
DataSource sub-classes.
update_events is only ever referenced on an instance of DataSource (or a
sub-class); no code relies on it being a class attribute. (In fact,
it's only used within methods on DataSource or its sub-classes, so it
doesn't even _need_ to remain public, though I think it's appropriate
for it to be public.)
DataSourceScaleway is also updated to move update_events from a
class attribute to an instance attribute, as the class attribute would
now be masked by the DataSource instance attribute.
LP: #1819913
|
|
When the network configuration has a default route configured and
another network device that is configured with dhcp, SUSE sysconfig
output should not accept the default route provided by the dhcp
server.
LP: #1812117
|
|
Our previous understanding of the upgrade issue was incomplete; it turns
out the only change we need is the one now outlined.
|
|
In some cases, the 'passwd' command might not be available, but
'usermod' might be. In debian systems both are provided by the
'passwd' package. In Redhat/Centos passwd comes from 'passwd' package
while 'usermod' comes from `shadow-utils`
This should just support either one with no real cost other than
the check.
|
|
Previously, init.paths.cloud_dir has a trailing slash, which meant that
"/var/lib/cloud//seed" was being compared to "/var/lib/cloud/seed" and
(of course), never matching.
In this commit, switch to using os.path.join to avoid this case (and
update the tests to catch it in future).
LP: #1818571
|
|
Some deployments of OpenStack expose link types to the guest which
cloud-init doesn't recognise. These will almost always be physical, so
we can operate more robustly if we assume that they are (whilst warning
the user that we're seeing something unexpected).
LP: #1639263
|
|
- Remove the last few places that use `if PY26`
- Replace our Python version detection logic with six's (which we were
already using in most places)
|
|
When providing netplan configuration to cloud-init, the internal
network state would enable DHCP if the 'dhcp' key was present in
the source config. In netplan, dhcp[46] is a boolean and the
value of the boolean should control whether DHCP is enabled rather
than the presence of the key. This issue leaded to inconsistant
sysconfig/network-scripts on fedora. 'BOOTPROTO' was always 'dhcp',
even if the address config was static.
After this change a dhcp subnet is added only if the 'dhcp' setting
in source cfg dict is True.
LP: #1818032
|
|
This was introduced due to Ubuntu using S3 mirrors, and S3 having a
buggy pipelining implementation. Those Ubuntu mirrors are no longer in
production and, furthremore, apt has also grown the ability to handle
servers with broken pipelining.
As such, we can stop disabling pipelining, which should result in
improved apt download speeds.
LP: #1794982
|
|
In test_ds_identify, don't mutate otherwise-static test data. When
running tests in a random order, this was causing failures due to
breaking preconditions for other tests.
In tests/helpers, reset logging level in tearDown. Some of the CLI
tests set the level of the root logger in a way that isn't correctly
reset.
For test_poll_imds_re_dhcp_on_timeout and
test_dhcp_discovery_run_in_sandbox_warns_invalid_pid, mock out
time.sleep; this saves ~11 seconds (or ~40% of previous test time!).
|
|
six already provides this for us, and we're already paying the cost to
determine it there; no need to do it twice.
|
|
Under Python 3.7, we are seeing `FutureWarning: Possible nested set at
position 23`; escaping this bracket causes that warning to disappear.
LP: #1816967
|
|
Encrypted data bags require a secrets file to be present to
decrypt, and the location of the file must be configured the
Chef client configuration file, client.rb.
This update enables cloud-init's chef module to update that
setting in client.rb.
LP: #1817082
|
|
The Azure data source is expected to expose a list of
ssh keys for the user-to-be-provisioned in the crawled
metadata. When configured to use the __builtin__ agent
this list is built by the WALinuxAgentShim. The shim
retrieves the full set of certificates and public keys
exposed to the VM from the wireserver, extracts any
ssh keys it can, and returns that list.
This fix reduces that list of ssh keys to just the
ones whose fingerprints appear in the "administrative
user" section of the ovf-env.xml file. The Azure
control plane exposes other ssh keys to the VM for
other reasons, but those should not be added to the
authorized_keys file for the provisioned user.
|
|
AWS EC2 instances' network come in 2 basic flavors: Classic and VPC
(Virtual Private Cloud). The former has an interesting behavior of having
its MAC address changed whenever the instance is stopped/restarted. This
behavior is not observed in VPC instances.
In Ubuntu 18.04 (Bionic) the network "management" changed from ENI-style
(etc/network/interfaces) to netplan, and when using netplan we observe
the following block present in /etc/netplan/50-cloud-init.yaml:
match:
macaddress: aa:bb:cc:dd:ee:ff
Jani Ollikainen noticed in Launchpad bug #1802073 that the EC2 Classic
instances were booting without network access in Bionic after stop/restart
procedure, due to their MAC address change behavior. It was narrowed down
to the netplan MAC match block, that kept the old MAC address after
stopping and restarting an instance, since the network configuration
writing happens by default only once in EC2 instances, in the first boot.
This patch changes the network configuration write to every boot in EC2
Classic instances, by checking against the "vpc-id" metadata information
provided only in the VPC instances - if we don't have this metadata value,
cloud-init will rewrite the network configuration file in every boot.
This was tested in an EC2 Classic instance and proved to fix the issue;
unit tests were also added for the new method is_classic_instance().
LP: #1802073
Reported-by: Jani Ollikainen <jani.ollikainen@ik.fi>
Suggested-by: Ryan Harper <ryan.harper@canonical.com>
Co-developed-by: Chad Smith <chad.smith@canonical.com>
Signed-off-by: Guilherme G. Piccoli <gpiccoli@canonical.com>
|
|
FreeBSD ifconfig output for ipv6 addrs doesn't find scopeid values
when present in the output and the pformat rendering assumes that
an ipv6 address will have a 'scope6' entry in the netdev info
dictionary. This patch finds the scopeid value, which is not
always inside <>, and in some cases v6 addrs don't have a scopeid
value in the output, so when rendering the table, allow scope6 value
to be replaced with the empty value.
LP: #1779672
|
|
Cloud-init rendered netplan with duplicate aliases if a network config
included "global" nameserver/search values. Netplan uses can read yaml
files which do use aliaes but cloud-init did not render a single yaml
dictionary, instead it combined yaml sections into a single document
which sometimes resulted in duplicate aliases being present.
This branch introduces a yaml SafeDumper class which can set the
'ignore_aliases' attribute. This is not enabled by default but callers
to util.yaml_dumps can pass a boolean to toggle this. The netplan
render uses noalias=True and the resulting yaml output does not contain
any aliases.
LP: #1815051
|
|
Avoid traceback when cloud-init clean is run from within
/var/lib/cloud/ deleted dirs.
LP: #1795508
|
|
Fixes:
- flake8: use ==/!= to compare str, bytes, and int literals
- pycodestyle: E117 over-indented
|
|
In addition to EPOCHREALTIME there is also an EPOCHSECONDS environment
variable that OpenNebula needs to exclude as it is expected to change.
This commit supplements the other exclusion in commit
d1a2fe7307e9cf2251d1f9a666c12d71d3f522d6.
Without this fix, unittests will intermittently fail if
parse_shell_config is run across a timing boundary where the
EPOCHSECONDS changes mid-test.
LP: #1813641
|
|
ONBOOT is not recognized on openSUSE and SUSE Linux Enterprise,
add the STARTMODE setting
LP: #1799540
|
|
|
|
This branch is needed to allow cloud-init to sbuild on Ubuntu Disco.
OpenNebula:parse_shell_config tries to do a comparison of bash
environment values, excluding expected environment variables which
are known to change.
Bash on Ubuntu Disco surfaces a new EPOCHREALTIME environment variable
which wasn't in previous bash environments, this var needs to be
ignored by parse_shell_config too.
LP: #1813383
|
|
When using the LXD module cloud-init will attempt
to install ZFS if it does not exist on the target
system. However instead of installing the `zfsutils-linux`
package it attempts to install `zfs` resulting in an error.
Ubuntu Xenial (16.04) has zfs meta package, but Bionic (18.04)
does not. Use the specific base package instead of zfs meta.
Co-authored-by: Michael Skalka <michael.skalka@canonical.com>
LP: #1799779
|
|
Writing the file with no dns information may prevent distro tools
from writing a resolv.conf file with dns information obtained from
a dhcp server.
|
|
The 'sysconfig' renderer is activated if, and only if, there's ifup and
ifdown commands present in its search dictonary or the network-scripts
configuration files are found. This patch adds a check for Network-
Manager configuration file as well.
This solution is based on the use of the plugin 'ifcfg-rh' present in
Network-Manager and is designed to support Fedora 29 or other
distributions that also replaced network-scripts by Network-Manager.
|
|
Correct invalid regex to match hashes starting with the following:
- $1, $2a, $2y, $5 or $6
LP: #1811446
|
|
cloud-init uses dhclient to fetch the DHCP lease so it can extract
DHCP options. dhclient creates the leasefile, then writes to it;
simply waiting for the leasefile to appear creates a race between
dhclient and cloud-init. Instead, wait for dhclient to be parented by
init. At that point, we know it has written to the leasefile, so it's
safe to copy the file and kill the process.
cloud-init creates a temporary directory in which to execute dhclient,
and deletes that directory after it has killed the process. If
cloud-init abandons waiting for dhclient to daemonize, it will still
attempt to delete the temporary directory, but will not report an
exception should that attempt fail.
LP: #1794399
|
|
Testing startup of large numbers of VMs (of varying distros) in Azure
shows that 3 retries results in a small percentage of failed VMs.
Increasing that by a few dramatically decreases the occurrence of
provisioning timeout errors. The initial choice of "3 retries" was
uninformed by heavy testing. Also, the alternate provisioning
mechanism for Azure (waagent) retries the Wireserver crawl without
limit. 10 retries seems a more reasonable choice.
|
