| Age | Commit message (Collapse) | Author |
|---|
|
Instead of logging the token values used log the headers and replace the actual
values with the string 'REDACTED'. This allows users to examine cloud-init.log
and see that the IMDSv2 token header is being used but avoids leaving the value
used in the log file itself.
LP: #1863943
|
|
|
|
* cloudinit: replace "import mock" with "from unittest import mock"
* test-requirements.txt: drop mock
Co-authored-by: Chad Smith <chad.smith@canonical.com>
|
|
Make sure network_config is created when self._network_config is unset.
Co-authored-by: Scott Moser <smoser@brickies.net>
|
|
|
|
|
|
Azure stores the instance ID with an incorrect byte ordering for the
first three hyphen delimited parts. This results in invalid
is_new_instance checks forcing Azure datasource to recrawl the metadata
service.
When persisting instance-id from the metadata service, swap the
instance-id string byte order such that it is consistent with
that returned by dmi information. Check whether the instance-id
string is a byte-swapped match when determining correctly whether
the Azure platform instance-id has actually changed.
|
|
Headers param was accidentally omitted and no longer passed through to
readurl due to a previous commit.
To avoid this omission of params in the future, drop positional param
definitions from read_file_or_url and pass all kwargs through to readurl
when we are not operating on a file.
In util:read_seeded, correct the case where invalid positional param
file_retries was being passed into read_file_or_url.
Also drop duplicated file:// prefix addition from read_seeded because
read_file_or_url does that work anyway.
LP: #1854084
|
|
* ec2: Add support for AWS IMDS v2 (session-oriented)
AWS now supports a new version of fetching Instance Metadata[1].
Update cloud-init's ec2 utility functions and update ec2 derived
datasources accordingly. For DataSourceEc2 (versus ec2-look-alikes)
cloud-init will issue the PUT request to obtain an API token for
the maximum lifetime and then all subsequent interactions with the
IMDS will include the token in the header.
If the API token endpoint is unreachable on Ec2 platform, log a
warning and fallback to using IMDS v1 and which does not use
session tokens when communicating with the Instance metadata
service.
We handle read errors, typically seen if the IMDS is beyond one
etwork hop (IMDSv2 responses have a ttl=1), by setting the api token
to a disabled value and then using IMDSv1 paths.
To support token-based headers, ec2_utils functions were updated
to support custom headers_cb and exception_cb callback functions
so Ec2 could store, or refresh API tokens in the event of token
becoming stale.
[1] https://docs.aws.amazon.com/AWSEC2/latest/ \
UserGuide/ec2-instance-metadata.html \
#instance-metadata-v2-how-it-works
|
|
The cloudsigma tests had few test cases that were not getting
all the "mocks" set up correctly. Specifically is_running_in_cloudsigma
was not getting replaced and calls would leak through to
util.read_dmi_data.
|
|
Router advertisements are required for the default route
to be set up, thus accept_ra should be enabled for
dhcpv6-stateful.
sysconf: IPV6_FORCE_ACCEPT_RA controls accept_ra sysctl.
eni: mode static and mode dhcp 'accept_ra' controls sysctl.
Add 'accept-ra: true|false' parameter to config v1 and
v2. When True: accept_ra is set to '1'. When False:
accept_ra is set to '0'. When not defined in config the
value is left to the operating system default.
This change also extend the IPv6 support to distinguish
between slaac and dhcpv6-stateless. SLAAC is autoconfig
without any options from DHCP, while stateless auto-configures
the address and the uses DHCP for other options.
LP: #1806014
LP: #1808647
|
|
Azure's Instance Metadata Service (IMDS) reports multiple IPv6
addresses, via the http://169.254.169.254/metadata/instance/network
route. Any additional values after the first in 'ipAddresses' under the
'ipv6' interface key are extracted and configured as static IPs on
the interface.
|
|
Network v2 configuration for Azure will set both dhcp4 and
dhcp6 to False by default.
When IPv6 privateIpAddresses are present for an interface in Azure's
Instance Metadata Service (IMDS), set dhcp6: True and provide a
route-metric value that will match the corresponding dhcp4 route-metric.
The route-metric value will increase by 100 for each additional
interface present to ensure the primary interface has a route to IMDS.
Also fix dhcp route-metric rendering for eni and sysconfig distros.
LP: #1850308
|
|
When ConfigDrive discovers the source path /config-drive, subplatform
is now reports 'config-drive'
LP: #1849731
|
|
In typical cases, SmartOS does not use DHCP for network configuration.
As such, if the network configuration changes that is reflected in
metadata and will be picked up during the next boot.
LP: #1765801
Joyent: OS-6902 reconfigure network on each boot
|
|
For security concern, we disable the custom script
by default.If a custom script is provided, stop
customization unless the custom script is explicitly
enabled by tools config.
|
|
After initial boot ovf-env.xml is copied to agent dir
(/var/lib/waagent/) with REDACTED password.
On subsequent boots DataSourceAzure loads with a configuration where the
user specified in /var/lib/waagent/ovf-env.xml is locked.
If instance id changes, cc_users_groups action will lock the user.
Fix this behavior by not locking the user if its password is REDACTED.
LP: #1849677
|
|
Openstack subnets can be configured to use SLAAC by setting
ipv6_address_mode=dhcpv6-stateless. When this is the case
the sysconfig interface configuration should use
IPV6_AUTOCONF=yes and not set DHCPV6C=yes.
This change sets the subnets type property to the full
network['type'] from openstack metadata.
cloudinit/net/sysconfig.py and cloudinit/net/eni.py
are updated to support new subnet types:
- 'ipv6_dhcpv6-stateless' => IPV6_AUTOCONF=yes
- 'ipv6_dhcpv6-stateful' => DHCPV6C=yes
Type 'dhcp6' in sysconfig is kept for backward compatibility
with any implementations that set subnet_type == 'dhcp6'.
LP: #1847517
|
|
e24cloud provides an EC2 compatible datasource.
This just identifies their platform based on dmi 'system-vendor'
having 'e24cloud'. https://www.e24cloud.com/en/ .
Updated chassis typo in zstack unit test docstring.
LP: #1696476
|
|
|
|
Zstack platform provides a AWS Ec2 metadata service, and
identifies their platform to the guest by setting the 'chassis asset tag'
to a string that ends with '.zstack.io'.
LP: #1841181
|
|
VMWware customization already has support to run a custom script during
the VM customization. Adding this option allows a VM administrator to
disable the execution of customization scripts. If set the script
will not execute and the customization status is set to
GUESTCUST_ERROR_SCRIPT_DISABLED.
|
|
Make sure Exoscale supplements or overrides existing system config
setting cloud_config_modules instead of replacing it with a one item
list set-passords
LP: #1841454
|
|
Collect and record the following information through KVP:
+ timestamps related to kernel initialization and systemd activation
of cloud-init services
+ system information including cloud-init version, kernel version,
distro version, and python version
+ diagnostic events for the most common provisioning error issues
such as empty dhcp lease, corrupted ovf-env.xml, etc.
+ increasing the log frequency of polling IMDS during reprovision.
|
|
The function generate_fallback_config is used by Azure by default when
not consuming IMDS configuration data. This function is also used by any
datasource which does not implement it's own network config. This simple
fallback configuration sets up dhcp on the most likely NIC. It will now
emit network v2 instead of network v1.
This is a step toward moving all components talking in v2 and allows us
to avoid costly conversions between v1 and v2 for newer distributions
which rely on netplan.
|
|
This adds an empty publish_host_keys() method to the default datasource
that is called by cc_ssh.py. This feature can be controlled by the
'ssh_publish_hostkeys' config option. It is enabled by default but can
be disabled by setting 'enabled' to false. Also, a blacklist of key
types is supported.
In addition, this change implements ssh_publish_hostkeys() for the GCE
datasource, attempting to write the hostkeys to the instance's guest
attributes. Using these hostkeys for ssh connections is currently
supported by the alpha version of Google's 'gcloud' command-line tool.
(On Google Compute Engine, this feature will be enabled by setting the
'enable-guest-attributes' metadata key to 'true' for the
project/instance that you would like to use this feature for. When
connecting to the instance for the first time using 'gcloud compute ssh'
the hostkeys will be read from the guest attributes for the instance and
written to the user's local known_hosts file for Google Compute Engine
instances.)
|
|
- dsidentify switches to the new Exoscale datasource on matching DMI name
- New Exoscale datasource added
Signed-off-by: Mathieu Corbin <mathieu.corbin@exoscale.ch>
|
|
Currently, if a platform provides any network configuration via the
"cmdline" method (i.e. network-data=... on the kernel command line,
ip=... on the kernel command line, or iBFT config via /run/net-*.conf),
the value of the data source's network_config property is completely
ignored.
This means that on platforms that use iSCSI boot (such as Oracle Compute
Infrastructure), there is no way for the data source to configure any
network interfaces other than those that have already been configured by
the initramfs.
This change allows data sources to specify the order in which network
configuration sources are considered. Data sources that opt to use this
mechanism will be expected to consume the command line network data and
integrate it themselves.
(The generic merging of network configuration sources was considered,
but we concluded that the single use case we have presently (a) didn't
warrant the increased complexity, and (b) didn't give us a broad enough
view to be sure that our generic implementation would be sufficiently
generic. This change in no way precludes a merging strategy in future.)
|
|
The EphemeralDHCP context manager did not parse or handle
rfc3442 classless static routes which prevented reading
datasource metadata in some clouds. This branch adds support
for extracting the field from the leases output, parsing the
format and then adding the required iproute2 ip commands to
apply (and teardown) the static routes.
LP: #1821102
|
|
This allows cloud-init query region to show valid region data for Azure
|
|
- UFS file system support
- GPT partition table support
- add support for newfs's -L parameter (label)
- move freebsd specific test from Azure to freebsd
|
|
blkid is a Linux-only command. With this patch, cloud-init uses another
approach to find the data source on FreeBSD.
LP: #1645824
|
|
The Azure data source helper attempts to use information in the dhcp
lease to find the Wireserver endpoint (IP address). Under some unusual
circumstances, those attempts will fail. This change uses a static
address, known to be always correct in the Azure public and sovereign
clouds, when the helper fails to locate a valid dhcp lease. This
address is not guaranteed to be correct in Azure Stack environments;
it's still best to use the information from the lease whenever possible.
|
|
If the IMDS primary server is not available, falling back to the
secondary server takes about 1s. The net result is that the
expected E2E time is slightly more than 1s. This change increases
the timeout to 2s to prevent the infinite loop of timeouts.
|
|
Mock util.SeLinuxGuard to do nothing within tests that mock functions
used by the guard, when those mocks confuse the guard. This has no
impact when executing unit tests on systems which do not enable selinux
(e.g. Ubuntu).
LP: #1825253
|
|
Moving update_events from a class attribute to an instance attribute
means that it doesn't exist on DataSource objects that are unpickled,
causing tracebacks on cloud-init upgrade.
As this change is only required for cloud-init installations which don't
utilise ds-identify, we're backing it out to be reintroduced once the
upgrade path bug has been addressed.
This reverts commit f2fd6eac4407e60d0e98826ab03847dda4cde138.
|
|
NoCloud data source now accepts both 'cidata' and 'CIDATA'
as filesystem labels. This is similar to DataSourceConfigDrive's
support for 'config-2' and 'CONFIG-2'.
|
|
The Azure platform surfaces random bytes into /sys via Hyper-V.
Python 2.7 json.dump() raises an exception if asked to convert
a str with non-character content, and python 3.0 json.dump()
won't serialize a "bytes" value. As a result, c-i instance
data is often not written by Azure, making reboots slower (c-i
has to repeat work).
The random data is base64-encoded and then decoded into a string
(str or unicode depending on the version of Python in use). The
base64 string has just as many bits of entropy, so we're not
throwing away useful "information", but we can be certain
json.dump() will correctly serialize the bits.
|
|
Currently, DataSourceAzure updates self.update_events in __init__. As
update_events is a class attribute on DataSource, this updates it for
all instances of classes derived from DataSource including those for
other clouds. This means that if DataSourceAzure is even instantiated,
its behaviour is applied to whichever data source ends up being used for
boot.
To address this, update_events is moved from a class attribute to an
instance attribute (that is therefore populated at instantiation time).
This retains the defaults for all DataSource sub-class instances, but
avoids them being able to mutate the state in instances of other
DataSource sub-classes.
update_events is only ever referenced on an instance of DataSource (or a
sub-class); no code relies on it being a class attribute. (In fact,
it's only used within methods on DataSource or its sub-classes, so it
doesn't even _need_ to remain public, though I think it's appropriate
for it to be public.)
DataSourceScaleway is also updated to move update_events from a
class attribute to an instance attribute, as the class attribute would
now be masked by the DataSource instance attribute.
LP: #1819913
|
|
Some deployments of OpenStack expose link types to the guest which
cloud-init doesn't recognise. These will almost always be physical, so
we can operate more robustly if we assume that they are (whilst warning
the user that we're seeing something unexpected).
LP: #1639263
|
|
- Remove the last few places that use `if PY26`
- Replace our Python version detection logic with six's (which we were
already using in most places)
|
|
pylint now complains about assignment of None from a
return of a function call. This does not account for subclassing
so we resolve this issue by removing the assignment in the unittest.
|
|
In test_ds_identify, don't mutate otherwise-static test data. When
running tests in a random order, this was causing failures due to
breaking preconditions for other tests.
In tests/helpers, reset logging level in tearDown. Some of the CLI
tests set the level of the root logger in a way that isn't correctly
reset.
For test_poll_imds_re_dhcp_on_timeout and
test_dhcp_discovery_run_in_sandbox_warns_invalid_pid, mock out
time.sleep; this saves ~11 seconds (or ~40% of previous test time!).
|
|
The Azure data source is expected to expose a list of
ssh keys for the user-to-be-provisioned in the crawled
metadata. When configured to use the __builtin__ agent
this list is built by the WALinuxAgentShim. The shim
retrieves the full set of certificates and public keys
exposed to the VM from the wireserver, extracts any
ssh keys it can, and returns that list.
This fix reduces that list of ssh keys to just the
ones whose fingerprints appear in the "administrative
user" section of the ovf-env.xml file. The Azure
control plane exposes other ssh keys to the VM for
other reasons, but those should not be added to the
authorized_keys file for the provisioned user.
|
|
AWS EC2 instances' network come in 2 basic flavors: Classic and VPC
(Virtual Private Cloud). The former has an interesting behavior of having
its MAC address changed whenever the instance is stopped/restarted. This
behavior is not observed in VPC instances.
In Ubuntu 18.04 (Bionic) the network "management" changed from ENI-style
(etc/network/interfaces) to netplan, and when using netplan we observe
the following block present in /etc/netplan/50-cloud-init.yaml:
match:
macaddress: aa:bb:cc:dd:ee:ff
Jani Ollikainen noticed in Launchpad bug #1802073 that the EC2 Classic
instances were booting without network access in Bionic after stop/restart
procedure, due to their MAC address change behavior. It was narrowed down
to the netplan MAC match block, that kept the old MAC address after
stopping and restarting an instance, since the network configuration
writing happens by default only once in EC2 instances, in the first boot.
This patch changes the network configuration write to every boot in EC2
Classic instances, by checking against the "vpc-id" metadata information
provided only in the VPC instances - if we don't have this metadata value,
cloud-init will rewrite the network configuration file in every boot.
This was tested in an EC2 Classic instance and proved to fix the issue;
unit tests were also added for the new method is_classic_instance().
LP: #1802073
Reported-by: Jani Ollikainen <jani.ollikainen@ik.fi>
Suggested-by: Ryan Harper <ryan.harper@canonical.com>
Co-developed-by: Chad Smith <chad.smith@canonical.com>
Signed-off-by: Guilherme G. Piccoli <gpiccoli@canonical.com>
|
|
The change here will utilize ssh keys found inside an instance's tag.
The tag value must start with 'AUTHORIZED_KEY'.
|
|
Transport functions (transport_iso9660 and transport_vmware_guestinfo)
would return a tuple of 3 values, but only the first was ever used
outside of test. The other values (device and filename) were just
ignored.
This just simplifies the transport functions to now return content
(in string format) or None indicating that the transport was not found.
|
|
This adds support for reading OVF information over the
'com.vmware.guestInfo' tranport. The current implementation requires
vmware-rpctool be installed in the system.
LP: #1807466
|
|
NoCloud's 'network-config' file was originally expected to contain
network configuration without the top level 'network' key. This was
because the file was named 'network-config' so specifying 'network'
seemed redundant.
However, JuJu is currently providing a top level 'network' config when
it tries to disable networking ({"network": {"config": "disabled"}).
Other users have also been surprised/confused by the fact that
a network config in /etc/cloud/cloud.cfg.d/network.cfg differed from
what was expected in 'network-config'.
LP: #1798117
|
|
Replace Azure pre-provision polling on IMDS with a blocking call
which watches for netlink link state change messages. The media
change event happens when a pre-provisioned VM has been activated
and is connected to the users virtual network and cloud-init can
then resume operation to complete image instantiation.
|
