| Age | Commit message (Collapse) | Author |
|---|
|
six already provides this for us, and we're already paying the cost to
determine it there; no need to do it twice.
|
|
Under Python 3.7, we are seeing `FutureWarning: Possible nested set at
position 23`; escaping this bracket causes that warning to disappear.
LP: #1816967
|
|
Encrypted data bags require a secrets file to be present to
decrypt, and the location of the file must be configured the
Chef client configuration file, client.rb.
This update enables cloud-init's chef module to update that
setting in client.rb.
LP: #1817082
|
|
The Azure data source is expected to expose a list of
ssh keys for the user-to-be-provisioned in the crawled
metadata. When configured to use the __builtin__ agent
this list is built by the WALinuxAgentShim. The shim
retrieves the full set of certificates and public keys
exposed to the VM from the wireserver, extracts any
ssh keys it can, and returns that list.
This fix reduces that list of ssh keys to just the
ones whose fingerprints appear in the "administrative
user" section of the ovf-env.xml file. The Azure
control plane exposes other ssh keys to the VM for
other reasons, but those should not be added to the
authorized_keys file for the provisioned user.
|
|
AWS EC2 instances' network come in 2 basic flavors: Classic and VPC
(Virtual Private Cloud). The former has an interesting behavior of having
its MAC address changed whenever the instance is stopped/restarted. This
behavior is not observed in VPC instances.
In Ubuntu 18.04 (Bionic) the network "management" changed from ENI-style
(etc/network/interfaces) to netplan, and when using netplan we observe
the following block present in /etc/netplan/50-cloud-init.yaml:
match:
macaddress: aa:bb:cc:dd:ee:ff
Jani Ollikainen noticed in Launchpad bug #1802073 that the EC2 Classic
instances were booting without network access in Bionic after stop/restart
procedure, due to their MAC address change behavior. It was narrowed down
to the netplan MAC match block, that kept the old MAC address after
stopping and restarting an instance, since the network configuration
writing happens by default only once in EC2 instances, in the first boot.
This patch changes the network configuration write to every boot in EC2
Classic instances, by checking against the "vpc-id" metadata information
provided only in the VPC instances - if we don't have this metadata value,
cloud-init will rewrite the network configuration file in every boot.
This was tested in an EC2 Classic instance and proved to fix the issue;
unit tests were also added for the new method is_classic_instance().
LP: #1802073
Reported-by: Jani Ollikainen <jani.ollikainen@ik.fi>
Suggested-by: Ryan Harper <ryan.harper@canonical.com>
Co-developed-by: Chad Smith <chad.smith@canonical.com>
Signed-off-by: Guilherme G. Piccoli <gpiccoli@canonical.com>
|
|
FreeBSD ifconfig output for ipv6 addrs doesn't find scopeid values
when present in the output and the pformat rendering assumes that
an ipv6 address will have a 'scope6' entry in the netdev info
dictionary. This patch finds the scopeid value, which is not
always inside <>, and in some cases v6 addrs don't have a scopeid
value in the output, so when rendering the table, allow scope6 value
to be replaced with the empty value.
LP: #1779672
|
|
Cloud-init rendered netplan with duplicate aliases if a network config
included "global" nameserver/search values. Netplan uses can read yaml
files which do use aliaes but cloud-init did not render a single yaml
dictionary, instead it combined yaml sections into a single document
which sometimes resulted in duplicate aliases being present.
This branch introduces a yaml SafeDumper class which can set the
'ignore_aliases' attribute. This is not enabled by default but callers
to util.yaml_dumps can pass a boolean to toggle this. The netplan
render uses noalias=True and the resulting yaml output does not contain
any aliases.
LP: #1815051
|
|
Avoid traceback when cloud-init clean is run from within
/var/lib/cloud/ deleted dirs.
LP: #1795508
|
|
Fixes:
- flake8: use ==/!= to compare str, bytes, and int literals
- pycodestyle: E117 over-indented
|
|
In addition to EPOCHREALTIME there is also an EPOCHSECONDS environment
variable that OpenNebula needs to exclude as it is expected to change.
This commit supplements the other exclusion in commit
d1a2fe7307e9cf2251d1f9a666c12d71d3f522d6.
Without this fix, unittests will intermittently fail if
parse_shell_config is run across a timing boundary where the
EPOCHSECONDS changes mid-test.
LP: #1813641
|
|
ONBOOT is not recognized on openSUSE and SUSE Linux Enterprise,
add the STARTMODE setting
LP: #1799540
|
|
|
|
This branch is needed to allow cloud-init to sbuild on Ubuntu Disco.
OpenNebula:parse_shell_config tries to do a comparison of bash
environment values, excluding expected environment variables which
are known to change.
Bash on Ubuntu Disco surfaces a new EPOCHREALTIME environment variable
which wasn't in previous bash environments, this var needs to be
ignored by parse_shell_config too.
LP: #1813383
|
|
When using the LXD module cloud-init will attempt
to install ZFS if it does not exist on the target
system. However instead of installing the `zfsutils-linux`
package it attempts to install `zfs` resulting in an error.
Ubuntu Xenial (16.04) has zfs meta package, but Bionic (18.04)
does not. Use the specific base package instead of zfs meta.
Co-authored-by: Michael Skalka <michael.skalka@canonical.com>
LP: #1799779
|
|
Writing the file with no dns information may prevent distro tools
from writing a resolv.conf file with dns information obtained from
a dhcp server.
|
|
The 'sysconfig' renderer is activated if, and only if, there's ifup and
ifdown commands present in its search dictonary or the network-scripts
configuration files are found. This patch adds a check for Network-
Manager configuration file as well.
This solution is based on the use of the plugin 'ifcfg-rh' present in
Network-Manager and is designed to support Fedora 29 or other
distributions that also replaced network-scripts by Network-Manager.
|
|
Correct invalid regex to match hashes starting with the following:
- $1, $2a, $2y, $5 or $6
LP: #1811446
|
|
cloud-init uses dhclient to fetch the DHCP lease so it can extract
DHCP options. dhclient creates the leasefile, then writes to it;
simply waiting for the leasefile to appear creates a race between
dhclient and cloud-init. Instead, wait for dhclient to be parented by
init. At that point, we know it has written to the leasefile, so it's
safe to copy the file and kill the process.
cloud-init creates a temporary directory in which to execute dhclient,
and deletes that directory after it has killed the process. If
cloud-init abandons waiting for dhclient to daemonize, it will still
attempt to delete the temporary directory, but will not report an
exception should that attempt fail.
LP: #1794399
|
|
Testing startup of large numbers of VMs (of varying distros) in Azure
shows that 3 retries results in a small percentage of failed VMs.
Increasing that by a few dramatically decreases the occurrence of
provisioning timeout errors. The initial choice of "3 retries" was
uninformed by heavy testing. Also, the alternate provisioning
mechanism for Azure (waagent) retries the Wireserver crawl without
limit. 10 retries seems a more reasonable choice.
|
|
The change here will utilize ssh keys found inside an instance's tag.
The tag value must start with 'AUTHORIZED_KEY'.
|
|
Transport functions (transport_iso9660 and transport_vmware_guestinfo)
would return a tuple of 3 values, but only the first was ever used
outside of test. The other values (device and filename) were just
ignored.
This just simplifies the transport functions to now return content
(in string format) or None indicating that the transport was not found.
|
|
This adds support for reading OVF information over the
'com.vmware.guestInfo' tranport. The current implementation requires
vmware-rpctool be installed in the system.
LP: #1807466
|
|
The tip-pylint tox target correctly reported the invalid use of
string formatting. The change here is to:
a.) Fix the error that was caught.
b.) move to pylint 2.2.2 for the default 'pylint' target.
|
|
Bump the version in cloudinit/version.py to be 18.5 and update ChangeLog.
LP: #1808380
|
|
It is possible to have a metric value in a per-subnet route.
This is currently missing in all renderers. Update each
renderer to emit the correct metric value from the config.
LP: #1805871
|
|
Add 'append: true' to write_files entries to append 'content' to file
specified by 'path' key. This modifies the file open mode to append.
|
|
I noticed a bug in dhclient_hook on the 'down' event, using 'is'
operator rather than '==' (if self.net_action is 'down').
This refactors/simplifies the code a bit for easier testing and adds
tests. The reason for the rename of 'action' to 'event' is to just
be internally consistent. The word and Namespace 'action' is used
by cloud-init main, so it was not really usable here.
Also adds a main which can easily be debugged with:
CI_DHCP_HOOK_DATA_D=./my.d python -m cloudinit.dhclient_hook up eth0
|
|
NoCloud's 'network-config' file was originally expected to contain
network configuration without the top level 'network' key. This was
because the file was named 'network-config' so specifying 'network'
seemed redundant.
However, JuJu is currently providing a top level 'network' config when
it tries to disable networking ({"network": {"config": "disabled"}).
Other users have also been surprised/confused by the fact that
a network config in /etc/cloud/cloud.cfg.d/network.cfg differed from
what was expected in 'network-config'.
LP: #1798117
|
|
Move routes under the nic's subnet rather than use top-level
("global") route config ensuring all net renderers will provide the
configured route.
Also updated cloudinit/cmd/devel/net_convert.py:
- Add input type 'vmware-imc' for OVF customization config files
- Fix bug when output-type was netplan which invoked netplan
generate/apply and attempted to write to
/etc/netplan/50-cloud-init.yaml instead of joining with the
output directory.
LP: #1806103
|
|
Replace Azure pre-provision polling on IMDS with a blocking call
which watches for netlink link state change messages. The media
change event happens when a pre-provisioned VM has been activated
and is connected to the users virtual network and cloud-init can
then resume operation to complete image instantiation.
|
|
Check the appropriate variables based on code review. Correcting what
seems to be a copy/paste mistake for the error handling from a few lines
above.
|
|
Since /run/cloud-init/instance-data-sensitive.json is root read-only,
ignore this file if non-root user runs collect-logs.
If --include-userdata is provided on the command line, exit in error
if non-root user attempts this operation.
Lastly, update the __main__ to exit based on return value of main.
LP: #1805201
|
|
We add a new Optional parameter: connectivity_url
This is used in __enter__ to verify if a connection already exists.
If it does exist, no operations are performed.
|
|
Upon URL timeout, _poll_imds is expected to re-dhcp to get updated
IP configuration. We don't want to indefinitely retry because the
instance likely has invalid IP configuration.
LP: #1803598
|
|
In some environments, like FreeBSD, gpart can return the device basename
instead of the full path. If this discovered devpath does not exist and
is missing the '/dev/' prefix, add that prefix in an attempt to find the
device.
|
|
There is an infrequent race when the booting instance can hit the IMDS
service before it is fully available. This results in a
requests.ConnectTimeout being raised.
Azure's retry_callback logic now retries on either 404s or Timeouts.
LP:1800223
|
|
If Azure detects an ntfs filesystem type during mount attempt, it should
still report the resource device as reformattable. There are slight
differences in error message format on RedHat and SuSE. This patch
simplifies the expected error match to work on both distributions.
LP: #1799338
|
|
In commitish 9073951 azure datasource tried to leverage stale DHCP
information obtained from EphemeralDHCPv4 context manager to report
updated provisioning status to the fabric earlier in the boot process.
Unfortunately the stale ephemeral network configuration had already been
torn down in preparation to bring up IMDS network config so the report
attempt failed on timeout.
This branch introduces obtain_lease and clean_network public methods on
EphemeralDHCPv4 to allow for setup and teardown of ephemeral network
configuration without using a context manager. Azure datasource now uses
this to persist ephemeral network configuration across multiple contexts
during provisioning to avoid multiple DHCP roundtrips.
|
|
There was a typo in the seeded filename s/azure-hotplug/hotplug-azure/.
|
|
When reusing a preprovisioned VM, report ready to Azure fabric as soon as
we get the reprovision data and the goal state so that we are not delayed
by the cloud-init stage switch, saving 2-3 seconds. Also reduce logging
when polling IMDS for reprovision data.
LP: #1799594
|
|
Emit a permissions error instead of "Missing instance-data.json" when
non-root user doesn't have read-permission on
/run/cloud-init/instance-data.json
|
|
On cloud-init upgrade path from 18.3 to 18.4 cloud-init changed how
instance-data is written. Cloud-init changes instance-data.json from root
read-only to redacted world-readable content, and provided a separate
unredacted instance-data-sensitive.json which is read-only root.
Since instance-data is only rewritten from cache on
reboot, the query and render tools needed fallback to use the 'old'
instance-data.json if the new sensitive file isn't yet present.
This avoids error messages from tools about an absebt
/run/instance-data-sensitive.json file.
LP: #1798189
|
|
openSUSE has changed the way the distribution is identified in
os-release. Add support detecting for openSUSE Leap 42.3, Leap 15
and TumbleWeed.
Reference: boo#1111427
|
|
Azure generates network configuration from the IMDS service and removes
any preexisting hotplug network scripts which exist in Azure cloud images.
Add a datasource configuration option which allows for writing a default
network configuration which sets up dhcp on eth0 and leave the hotplug
handling to the cloud-image scripts.
To disable network-config from Azure IMDS, add the following to
/etc/cloud/cloud.cfg.d/99-azure-no-imds-network.cfg:
datasource:
Azure:
apply_network_config: False
LP: #1798424
|
|
Fix a typo in a RuntimeError path. (udevamd -> udevadm).
|
|
Add a quick cloud lookup utility in order to more easily determine
the cloud on which an instance is running.
The utility parses standardized attributes from
/run/cloud-init/instance-data.json to print the canonical cloud-id
for the instance. It uses known region maps if necessary to determine
on which specific cloud the instance is running.
Examples:
aws, aws-gov, aws-china, rackspace, azure-china, lxd, openstack, unknown
|
|
Add the following instance-data.json standardized keys:
* v1._beta_keys: List any v1 keys in beta development,
e.g. ['subplatform'].
* v1.public_ssh_keys: List of any cloud-provided ssh keys for the
instance.
* v1.platform: String representing the cloud platform api supporting the
datasource. For example: 'ec2' for aws, aliyun and brightbox cloud
names.
* v1.subplatform: String with more details about the source of the
metadata consumed. For example, metadata uri, config drive device path
or seed directory.
To support the new platform and subplatform standardized instance-data,
DataSource and its subclasses grew platform and subplatform attributes.
The platform attribute defaults to the lowercase string datasource name at
self.dsname. This method is overridden in NoCloud, Ec2 and ConfigDrive
datasources.
The subplatform attribute calls a _get_subplatform method which will
return a string containing a simple slug for subplatform type such as
metadata, seed-dir or config-drive followed by a detailed uri, device or
directory path where the datasource consumed its configuration.
As part of this work, DatasourceEC2 methods _get_data and _crawl_metadata
have been refactored for a few reasons:
- crawl_metadata is now a read-only operation, persisting no attributes on
the datasource instance and returns a dictionary of consumed metadata.
- crawl_metadata now closely represents the raw stucture of the ec2
metadata consumed, so that end-users can leverage public ec2 metadata
documentation where possible.
- crawl_metadata adds a '_metadata_api_version' key to the crawled
ds.metadata to advertise what version of EC2's api was consumed by
cloud-init.
- _get_data now does all the processing of crawl_metadata and saves
datasource instance attributes userdata_raw, metadata etc.
Additional drive-bys:
* unit test rework for test_altcloud and test_azure to simplify mocks
and make use of existing util and test_helpers functions.
|
|
Previously we explicitly excluded mac address '00:00:00:00:00:00'.
But then some nics (tunl0 and sit0) ended up having a mac address like
'00:00:00:00'.
The change here just ignores all 00[:00[:00...]].
LP: #1796917
|
|
Bump the version in cloudinit/version.py to be 18.4 and update ChangeLog.
LP: #1795741
|
|
Relax expectation on path to lxc and lxd. The deb path still does
install them in /usr/bin/ but that is overly pedantic.
Add a 'lxd waitready' (present since lxd 0.5) to wait until lxd
is ready before operating on it.
|
