| Age | Commit message (Collapse) | Author |
|---|
|
Collect and record the following information through KVP:
+ timestamps related to kernel initialization and systemd activation
of cloud-init services
+ system information including cloud-init version, kernel version,
distro version, and python version
+ diagnostic events for the most common provisioning error issues
such as empty dhcp lease, corrupted ovf-env.xml, etc.
+ increasing the log frequency of polling IMDS during reprovision.
|
|
Oracle Cloud Infrastructure's Instance Metadata Service provides network
configuration information for non-primary NICs. This commit introduces
support, on Virtual Machines[0], for fetching that network metadata,
converting it to v1 network-config[1] and combining it into the network
configuration generated for the primary interface.
By default, this behaviour is not enabled. Configuring the Oracle
datasource to `configure_secondary_nics` enables it:
datasource:
Oracle:
configure_secondary_nics: true
Failures to fetch and generate secondary NIC configuration will log a
warning, but otherwise will not affect boot.
[0] The expected use of the IMDS-provided network configuration is
substantially different on Bare Metal Machines, so support for that
will be addressed separately.
[1] This is v1 config, because cloudinit.net.cmdline generates v1 config
and we need to integrate the secondary NICs into that configuration.
|
|
Building the subp arguments for a `useradd` call in a variable named
`adduser_cmd` is extremely confusing; let's not do that.
(This also changes the snap and freebsd variables to something more
apropos.)
|
|
The function generate_fallback_config is used by Azure by default when
not consuming IMDS configuration data. This function is also used by any
datasource which does not implement it's own network config. This simple
fallback configuration sets up dhcp on the most likely NIC. It will now
emit network v2 instead of network v1.
This is a step toward moving all components talking in v2 and allows us
to avoid costly conversions between v1 and v2 for newer distributions
which rely on netplan.
|
|
This adds an empty publish_host_keys() method to the default datasource
that is called by cc_ssh.py. This feature can be controlled by the
'ssh_publish_hostkeys' config option. It is enabled by default but can
be disabled by setting 'enabled' to false. Also, a blacklist of key
types is supported.
In addition, this change implements ssh_publish_hostkeys() for the GCE
datasource, attempting to write the hostkeys to the instance's guest
attributes. Using these hostkeys for ssh connections is currently
supported by the alpha version of Google's 'gcloud' command-line tool.
(On Google Compute Engine, this feature will be enabled by setting the
'enable-guest-attributes' metadata key to 'true' for the
project/instance that you would like to use this feature for. When
connecting to the instance for the first time using 'gcloud compute ssh'
the hostkeys will be read from the guest attributes for the instance and
written to the user's local known_hosts file for Google Compute Engine
instances.)
|
|
- dsidentify switches to the new Exoscale datasource on matching DMI name
- New Exoscale datasource added
Signed-off-by: Mathieu Corbin <mathieu.corbin@exoscale.ch>
|
|
What we had previously was inaccurate in a few respects.
LP: #1838794
|
|
Previously "cmdline" network configuration could be either
user-specified network-config=... configuration data, or
initramfs-provided configuration data. Before data sources could modify
the order in which network config sources were considered, this
conflation didn't matter (and, indeed, in the default data source
configuration it will continue to not matter).
However, it _is_ desirable for a data source to be able to specify that
its network configuration should be preferred over the
initramfs-provided network configuration but still allow explicit
network-config=... configuration passed to the kernel cmdline to
continue to override both of those sources.
(This also modifies the Oracle data source to use read_initramfs_config
directly, which is effectively what it was using
read_kernel_cmdline_config for previously.)
|
|
Currently, if a platform provides any network configuration via the
"cmdline" method (i.e. network-data=... on the kernel command line,
ip=... on the kernel command line, or iBFT config via /run/net-*.conf),
the value of the data source's network_config property is completely
ignored.
This means that on platforms that use iSCSI boot (such as Oracle Compute
Infrastructure), there is no way for the data source to configure any
network interfaces other than those that have already been configured by
the initramfs.
This change allows data sources to specify the order in which network
configuration sources are considered. Data sources that opt to use this
mechanism will be expected to consume the command line network data and
integrate it themselves.
(The generic merging of network configuration sources was considered,
but we concluded that the single use case we have presently (a) didn't
warrant the increased complexity, and (b) didn't give us a broad enough
view to be sure that our generic implementation would be sufficiently
generic. This change in no way precludes a merging strategy in future.)
|
|
If input to network_state.parse_net_config_data was netplan (v2 yaml)
then the network state would lose the mtu information on bond or vlan.
LP: #1836949
|
|
On systems with many interfaces, processing udev events may take a while.
Cloud-init expects devices included in a provided network-configuration
to be present when attempting to configure them. This patch adds a step
in net configuration where it will check for devices provided in the
configuration and if not found, issue udevadm settle commands to wait
for them to appear.
Additionally, the default path for udev persistent network rules
70-persistent-net.rules may also be written to systems which include
the 75-net-generator.rules. During boot, cloud-init and the
generator may race and interleave values causing issues. OpenSUSE
will now use a newer file, 85-persistent-net-cloud-init.rules which
will take precedence over values created by 75-net-generator and
avoid collisions on the same file.
LP: #1817368
|
|
Bump the version in cloudinit/version.py to be 19.2 and update ChangeLog.
LP: #1836921
|
|
The EphemeralDHCP context manager did not parse or handle
rfc3442 classless static routes which prevented reading
datasource metadata in some clouds. This branch adds support
for extracting the field from the leases output, parsing the
format and then adding the required iproute2 ip commands to
apply (and teardown) the static routes.
LP: #1821102
|
|
Support is for now implemented in such a way that it will fall back to
the old `_write_network()` if netplan is not available on the image.
|
|
cloud-init does not trigger reboots of a VM therefore adding custom
scripts to rc.local does not execute the post scripts. This patch
moves post-scripts into per-instance scripts dir and has cc_scripts
module run the post-scripts.
Also in this branch:
- Remove the sh interpreter and execute the customization script
directly.
- Update the unit test.
LP: #1833192
|
|
This branch introduces a new command line feature for cloud-init.
Currently, the cloud-init module has the capability to analyze events in
cloud-init.log in three ways: 'show', 'blame', 'dump'.
These changes add a fourth capability, called 'boot'.
Running the command 'cloud-init analyze boot' will provide the user three
timestamps.
1) Timestamp for when the kernel starts initializing.
2) Timestamp for when the kernel finishes its initialization.
3) Timestamp for when systemd activates cloud-init.
This feature enables cloud-init users to analyze different boot phases.
This would aid in debugging performance issues related
to cloud-init startup or tracking regression.
|
|
On Debian, ifupdown uses `source-directory /etc/network/interfaces.d`
(for new installs) to include files.
https://salsa.debian.org/debian/ifupdown/blob/master/debian/postinst#L23
The current filename, 50-cloud-init.cfg, does not match against the RE
that is used to scan the directory for configurations (ASCII upper- and
lower-case letters, ASCII digits, ASCII underscores, and ASCII
minus-hyphens):
https://salsa.debian.org/debian/ifupdown/blob/master/interfaces.5.pre#L122
Of course many installations use `source /etc/network/interfaces.d/*`,
but not all.
|
|
bonds may inherit mac address from a physical interface
LP: #1812857
|
|
* cc_lxd: fix copy/paste error in debug logging
* DataSourceCloudSigma: remove unreachable code
* This unreachable code was introduced in a refactor (in 2015) which
removed the need for an exception handler, but retained the logging
from the exception handler as an unreachable fall-through.
|
|
|
|
This allows cloud-init query region to show valid region data for Azure
|
|
Currently, only a few bonding parameters can be configured on
sysconfig systems. This patch aims to support more parameters
documented on the docs site.
|
|
|
|
Previous versions of netplan included a misspelling for the
bond parameter around gratuitous-arp. This has been fixed and released
and cloud-init needs to accept both values. This branch fixes the
key that will be rendered and transforms the previous misspelling
when capturing network_state.
LP: #1827238
|
|
- UFS file system support
- GPT partition table support
- add support for newfs's -L parameter (label)
- move freebsd specific test from Azure to freebsd
|
|
blkid is a Linux-only command. With this patch, cloud-init uses another
approach to find the data source on FreeBSD.
LP: #1645824
|
|
The Azure data source helper attempts to use information in the dhcp
lease to find the Wireserver endpoint (IP address). Under some unusual
circumstances, those attempts will fail. This change uses a static
address, known to be always correct in the Azure public and sovereign
clouds, when the helper fails to locate a valid dhcp lease. This
address is not guaranteed to be correct in Azure Stack environments;
it's still best to use the information from the lease whenever possible.
|
|
Bump the version on cloudinit/version.py to be 19.1 and update ChangeLog
LP: #1828479
|
|
Under some circumstances, cc_disk_setup may reformat volumes which
already appear in /etc/fstab (e.g. Azure ephemeral drive is reformatted
from NTFS to ext4 after service-heal). Normally, cc_mounts only calls
mount -a if it altered /etc/fstab. With this change cc_mounts will read
/proc/mounts and verify if configured mounts are already mounted and if
not raise flag to request a mount -a. This handles the case where no
changes to fstab occur but a mount -a is required due to change in
underlying device which prevented the .mount unit from running until
after disk was reformatted.
LP: #1825596
|
|
|
|
If the IMDS primary server is not available, falling back to the
secondary server takes about 1s. The net result is that the
expected E2E time is slightly more than 1s. This change increases
the timeout to 2s to prevent the infinite loop of timeouts.
|
|
+ Truncate KVP Pool file to prevent stale entries from
being processed by the Hyper-V KVP reporter.
+ Drop filtering of KVPs as it is no longer needed.
+ Batch appending of existing KVP entries.
|
|
Currently on 18.04, running tox -e py27 will spew errors like:
.tests/unittests/test_net.py:2649: YAMLLoadWarning: calling yaml.load()
without Loader=... is deprecated, as the default Loader is unsafe.
Please read https://msg.pyyaml.org/load for full details.
The change here just uses cloud-init's yaml, which does safeloading
by default.
|
|
Restrict the sysconfig renderer availabily to known distros.
Ubuntu/Debian systems may include network-manager but they do
not have support for reading sysconfig network output; that is
enabled via a Network-Manager plugin: ifcfg-rh which is not
available in Ubuntu/Debian.
LP: #1819994
|
|
On FreeBSD, mount_cd9660 does not accept the sync option that is enabled
by default. In addition, the sync is only useful with the `rw` mode.
However the `rw` mode was never used.
This patch removes the `rw` and `sync` parameter of `mount_cb` to
simplify the code base and resolve the FreeBSD issue.
LP: #1645824
|
|
|
|
Moving update_events from a class attribute to an instance attribute
means that it doesn't exist on DataSource objects that are unpickled,
causing tracebacks on cloud-init upgrade.
As this change is only required for cloud-init installations which don't
utilise ds-identify, we're backing it out to be reintroduced once the
upgrade path bug has been addressed.
This reverts commit f2fd6eac4407e60d0e98826ab03847dda4cde138.
|
|
NoCloud data source now accepts both 'cidata' and 'CIDATA'
as filesystem labels. This is similar to DataSourceConfigDrive's
support for 'config-2' and 'CONFIG-2'.
|
|
Cloud-init's main.py will fail when presented with a new
stage name 'modules-init' if upgrading an older cloud-init.
Fix this by initializing unknown stage names before accessing.
LP: #1815109
|
|
ubuntu-advantage-tools version 19 has a different command line
interface. Update cloud-init's config module to accept new
ubuntu_advantage configuration settings.
* Underscores better than hyphens: deprecate 'ubuntu-advantage'
cloud-config key in favor of 'ubuntu_advantage'
* Attach machines with either sso credentials of UA user_token
* Services are enabled by name though an 'enable' list
* Raise warnings if deprecated ubuntu-advantage config keys are
present, or errors if its config we cannott adapt to
Ubuntu Advantage support can now be configured via #cloud-config
with the following yaml:
ubuntu_advantage:
token: 'thisismyubuntuadvantagetoken'
enable: [esm, fips, livepatch]
Co-Authored-By: Daniel Watkins <daniel.watkins@canonical.com>
|
|
When the Azure datasource persists all of its metadata to the
instance directory, it deliberately sets the self.network_config
value to be the sources.UNSET value. The goal is to ensure that
each time the system boots, fresh network configuration data is
fetched from the cloud platform so that any control plane changes
will take effect. When a VM is first created, there's no pickled
instance to restore, so self._network_config is None, resulting
in self.network_config() properly building a new config. Azure
suffered from LP: #1801364 which prevented ds from being stored
in obj.pkl in the instance directory, so subsequent reboots always
regenerated their network configuration.
Commit 0dc3a77f41f4544e4cb5a41637af7693410d4cdf introduced a
new bug in which self.network_config() assumed the
self._network_config value was either None or trustable; when
the config was unpickled, that value was _unset, thus breaking
the assumption.
LP: #1823084
|
|
Create an Azure logging decorator and use additional ReportEventStack
context managers to provide additional logging details.
|
|
The Azure platform surfaces random bytes into /sys via Hyper-V.
Python 2.7 json.dump() raises an exception if asked to convert
a str with non-character content, and python 3.0 json.dump()
won't serialize a "bytes" value. As a result, c-i instance
data is often not written by Azure, making reboots slower (c-i
has to repeat work).
The random data is base64-encoded and then decoded into a string
(str or unicode depending on the version of Python in use). The
base64 string has just as many bits of entropy, so we're not
throwing away useful "information", but we can be certain
json.dump() will correctly serialize the bits.
|
|
For writing IPv6 addresses to ifcfg-* the name "IPV6ADDR" is used. For
secondary IPs the value for "IPV6ADDR_SECONDARIES" is set. On SUSE based
distributions the names "IPADDR6" and "IPADDR6_$SOMELABEL" need to be
used.
|
|
When rendering ipv6 static routes in eni format the
post-up/pre down commands were not correct for ipv6.
LP: #1818669
|
|
The ubuntu_drivers config module enables usage of the 'ubuntu-drivers'
command. At this point it only serves as a way of installing NVIDIA
drivers for general purpose graphics processing unit (GPGPU)
functionality.
Also, a small usability improvement to get_cfg_by_path to allow it to
take a string for the key path
"toplevel/second/mykey"
in addition to the original:
("toplevel", "second", "mykey")
|
|
Currently, DataSourceAzure updates self.update_events in __init__. As
update_events is a class attribute on DataSource, this updates it for
all instances of classes derived from DataSource including those for
other clouds. This means that if DataSourceAzure is even instantiated,
its behaviour is applied to whichever data source ends up being used for
boot.
To address this, update_events is moved from a class attribute to an
instance attribute (that is therefore populated at instantiation time).
This retains the defaults for all DataSource sub-class instances, but
avoids them being able to mutate the state in instances of other
DataSource sub-classes.
update_events is only ever referenced on an instance of DataSource (or a
sub-class); no code relies on it being a class attribute. (In fact,
it's only used within methods on DataSource or its sub-classes, so it
doesn't even _need_ to remain public, though I think it's appropriate
for it to be public.)
DataSourceScaleway is also updated to move update_events from a
class attribute to an instance attribute, as the class attribute would
now be masked by the DataSource instance attribute.
LP: #1819913
|
|
When the network configuration has a default route configured and
another network device that is configured with dhcp, SUSE sysconfig
output should not accept the default route provided by the dhcp
server.
LP: #1812117
|
|
Our previous understanding of the upgrade issue was incomplete; it turns
out the only change we need is the one now outlined.
|
|
In some cases, the 'passwd' command might not be available, but
'usermod' might be. In debian systems both are provided by the
'passwd' package. In Redhat/Centos passwd comes from 'passwd' package
while 'usermod' comes from `shadow-utils`
This should just support either one with no real cost other than
the check.
|
