| Age | Commit message (Collapse) | Author |
|---|
|
Also fix search path in networkd
|
|
In the nic attach path, we skip doing dhcp since we already did it
when bringing the interface up. However when polling for
reprovisiondata, it is possible for the request to timeout due to
platform issues. In those cases we still need to do dhcp and try again
since we tear down the context. We can only skip the first dhcp
attempt.
|
|
before rebinding again (#990)
Add 10 second polling loop in wait_for_link_up after performing
an unbind and re-bind of primary NIC in hv_netvsc driver.
Also reduce cloud-init logging levels to debug for these operations.
|
|
|
|
Add tests for cc_resolv_conf handler
|
|
* Replace broken httpretty tests with mock
Certain versions of python/httpretty don't work correctly using https
URIs. #960 recently added httpretty tests using https. This commit
replaces the httpretty tests that were failing on https with mocks of
readurl instead.
|
|
When bringing interface up by unbinding and then binding hv_netvsc
driver, it might take a short delay after binding for the link to be
up. So before trying unbind/bind again after sleep, check if the link
is up. This is a corner case when a preprovisioned VM is reused and
the NICs are hot-attached.
|
|
|
|
- update the puppet module to support AIO installations by setting
`install_type` to `aio`
- make the install collection configurable through the `collection`
parameter; by default the rolling `puppet` collection will be used,
which installs the latest version)
- when `install_type` is `aio`, puppetlabs repos will be purged after
installation; set `cleanup` to `False` to prevent this
- AIO installations are performed by downloading and executing a shell
script; the URL for this script can be overridden using the
`aio_install_url` parameter
- make it possible to run puppet agent after installation/configuration
via the `exec` key
- by default, puppet agent will run with the `--test` argument; this can
be overridden via the `exec_args` key
|
|
This patch finally introduces the Cloud-Init Datasource for VMware
GuestInfo as a part of cloud-init proper. This datasource has existed
since 2018, and rapidly became the de facto datasource for developers
working with Packer, Terraform, for projects like kube-image-builder,
and the de jure datasource for Photon OS.
The major change to the datasource from its previous incarnation is
the name. Now named DatasourceVMware, this new version of the
datasource will allow multiple transport types in addition to
GuestInfo keys.
This datasource includes several unique features developed to address
real-world situations:
* Support for reading any key (metadata, userdata, vendordata) both
from the guestinfo table when running on a VM in vSphere as well as
from an environment variable when running inside of a container,
useful for rapid dev/test.
* Allows booting with DHCP while still providing full participation
in Cloud-Init instance data and Jinja queries. The netifaces library
provides the ability to inspect the network after it is online,
and the runtime network configuration is then merged into the
existing metadata and persisted to disk.
* Advertises the local_ipv4 and local_ipv6 addresses via guestinfo
as well. This is useful as Guest Tools is not always able to
identify what would be considered the local address.
The primary author and current steward of this datasource spoke at
Cloud-Init Con 2020 where there was interest in contributing this datasource
to the Cloud-Init codebase.
The datasource currently lives in its own GitHub repository at
https://github.com/vmware/cloud-init-vmware-guestinfo. Once the datasource
is merged into Cloud-Init, the old repository will be deprecated.
|
|
|
|
In /etc/ssh/sshd_config, it is possible to define a custom
authorized_keys file that will contain the keys allowed to access the
machine via the AuthorizedKeysFile option. Cloudinit is able to add
user-specific keys to the existing ones, but we need to be careful on
which of the authorized_keys files listed to pick.
Chosing a file that is shared by all user will cause security
issues, because the owner of that key can then access also other users.
We therefore pick an authorized_keys file only if it satisfies the
following conditions:
1. it is not a "global" file, ie it must be defined in
AuthorizedKeysFile with %u, %h or be in /home/<user>. This avoids
security issues.
2. it must comply with ssh permission requirements, otherwise the ssh
agent won't use that file.
If it doesn't meet either of those conditions, write to
~/.ssh/authorized_keys
We also need to consider the case when the chosen authorized_keys file
does not exist. In this case, the existing behavior of cloud-init is
to create the new file. We therefore need to be sure that the file
complies with ssh permissions too, by setting:
- the actual file to permission 600, and owned by the user
- the directories in the path that do not exist must be root owned and
with permission 755.
|
|
|
|
Implement missing device_aliases feature
The device_aliases key has been documented as part of disk_setup for
years, however the feature was never implemented. This implements the
feature as documented allowing usercfg (rather than dsconfig) to create
a mapping of device names.
This is not to be confused with disk_aliases, a very similar map but
existing solely for use by datasources.
LP: #1867532
|
|
Currently cloud-init generates fallback network config on various
scenarios.
For example:
1. When no DS found
2. There is no 'network' info given in DS metadata.
3. If a DS gives a network config once and upon reboot if DS doesn't
give any network info, previously set network data will be
overridden.
A newly introduced key in cloud.cfg.tmpl can be used to control this
behavior on PhotonOS.
Also, if OS comes with a set of default network files(configs), like in
PhotonOS, cloud-init should not overwrite them by default.
This change also includes some nitpicking changes of reorganizing few
config variables.
Signed-off-by: Shreenidhi Shedi <sshedi@vmware.com>
|
|
Virtuozzo Linux is a distro based off of CentOS 8, similar to Alma Linux and Rocky Linux.
|
|
Details:
1. Support guest set network config through guestinfo.ovfEnv using OVF
2. 'network-config' Property is optional
3. 'network-config' Property's value has to be base64 encoded
Added unittests and updated ovf-env.xml example
|
|
In CI run against pylint 2.9.3 and fix occurrences of:
- W0237 (arguments-renamed)
- W0402 (deprecated-module)
The W0402 deprecated-module was about module `imp`:
cloudinit/patcher.py:9: [W0402(deprecated-module), ]
Uses of a deprecated module 'imp'
The imp module is deprecated and replaced by importlib, which according
to the documentation has no replacement for acquire_lock() and
release_lock(), which are the only reason why `imp` is imported.
Nothing about the code using this lock that actually requires it.
Let's remove the locking code and the import altogether.
Dropping the locking makes patcher.patch() an empty wrapper around
_patch_logging(). Rename _patch_logging() to patch_logging() and
call it directly instead. Drop patch().
|
|
With a few exceptions, Azure VM deployments receive provisioning
metadata through the provisioning iso presented as a cdrom device
(/dev/sr0). The existing code attempts to find this device by calling
blkid to find all devices that have either type iso9660 or udf. This
can be very expensive if the VM has a lot of disks. This commit will
attempt to mount the default iso location first and only tries to use
blkid to locate the iso location if the default mounting location fails
|
|
Adds a udev script which will invoke a hotplug hook script on all net
add events. The script will write some udev arguments to a systemd FIFO
socket (to ensure we have only instance of cloud-init running at a
time), which is then read by a new service that calls a new 'cloud-init
devel hotplug-hook' command to handle the new event.
This hotplug-hook command will:
- Fetch the pickled datsource
- Verify that the hotplug event is supported/enabled
- Update the metadata for the datasource
- Ensure the hotplugged device exists within the datasource
- Apply the config change on the datasource metadata
- Bring up the new interface (or apply global network configuration)
- Save the updated metadata back to the pickle cache
Also scattered in some unrelated typing where helpful
|
|
defined in AuthorizedKeysFile (#937)
This patch aims to fix LP1911680, by analyzing the files provided
in sshd_config and merge all keys into an user-specific file. Also
introduces additional tests to cover this specific case.
The file is picked by analyzing the path given in AuthorizedKeysFile.
If it points inside the current user folder (path is /home/user/*), it
means it is an user-specific file, so we can copy all user-keys there.
If it contains a %u or %h, it means that there will be a specific
authorized_keys file for each user, so we can copy all user-keys there.
If no path points to an user-specific file, for example when only
/etc/ssh/authorized_keys is given, default to ~/.ssh/authorized_keys.
Note that if there are more than a single user-specific file, the last
one will be picked.
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Co-authored-by: James Falcon <therealfalcon@gmail.com>
LP: #1911680
RHBZ:1862967
|
|
Add a new switch allow_raw_data to control raw data feature, update
the documentation. Fix bugs about max_wait.
|
|
We read the MTU from the subnet entries. With the v1 format, the MTU can
be set at the root level of the interface entry in the `config` section.
Limitation, we won't set the MTU if the interface use DHCP. This
would require a bit of refactoring.
Also simplify/clarify how we pass the target variable in `cloudinit.net.bsd`.
See: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256309
Reported-by: Andrey Fesenko
|
|
Currently _bring_up_interfaces() is a no-op for any distro using
renderers. We need to be able to support bringing up a single
interfaces, a list of interfaces, and all interfaces. This should be
independent of the renderers, as the network config is often
generated independent of the mechanism used to apply it.
Additionally, I included a refactor to remove
"_supported_write_network_config". We had a confusing call chain of
apply_network_config->_write_network_config->_supported_write_network_config.
The last two have been combined.
|
|
Minor fixes in networkd renderer & fixed corresponding tests
Removed datasource_list for Photon from cloud.cfg.tmpl & added a comment
in cloud.cfg.tmpl about not to use multiline array for datasource_list.
Signed-off-by: Shreenidhi Shedi <sshedi@vmware.com>
|
|
Also added a new (currently experimental) systemd-networkd renderer,
and includes a small refactor to cc_resolv_conf.py to support the
resolved.conf used by systemd-resolved.
|
|
- small document update for ReportEventStack explaining post_files
parameter
- small unit test for test_reporting demonstrating the close of an
event with optional post_files list
|
|
- Mostly based on FreeBSD, the main exception is that
`find_devs_with_on_freebsd` does not work.
- Since we cannot get the CDROM or the partition labels,
`find_devs_with_on_dragonflybsd()` has a more naive approach and
returns all the block devices.
|
|
instance-data.json redacts sensitive data for non-root users. Since user
data is consumed as root, we should be consuming the non-redacted data
instead.
LP: #1931392
|
|
Rocky Linux is a RHEL-compatible distribution so all changes that have
been made should be trivial.
|
|
Presently, mirror keys cannot be associated with primary/security
mirrors. Unfortunately, this prevents use of Landscape-managed
package mirrors as the mirror key for the Landscape-hosted repository
cannot be provided.
This patch allows the same key-related fields usable on "sources"
entries to be used on the "primary" and "security" entries as well.
LP: #1925395
|
|
Control is currently limited to boot events, though this should
allow us to more easily incorporate HOTPLUG support. Disabling
'instance-first-boot' is not supported as we apply networking config
too early in boot to have processed userdata (along with the fact
that this would be a pretty big foot-gun).
The concept of update events on datasource has been split into
supported update events and default update events. Defaults will be
used if there is no user-defined update events, but user-defined
events won't be supplied if they aren't supported.
When applying the networking config, we now check to see if the event
is supported by the datasource as well as if it is enabled.
Configuration looks like:
updates:
network:
when: ['boot']
|
|
AlmaLinux OS is RHEL-compatible so all the changes needed are trivial.
|
|
See https://bugs.launchpad.net/cloud-init/+bug/1910835
|
|
This reverts commit 74fa008bfcd3263eb691cc0b3f7a055b17569f8b.
During pre-release testing, we discovered two issues with this commit.
Firstly, there's a typo in the udevadm command that causes a TypeError
for _all_ growpart executions. Secondly, the LVM resizing does not
appear to successfully resize everything up to the LV, though some
things do get resized.
We certainly want this change, so we'll be happy to review and land it
alongside an integration test which confirms that it is working as
expected.
LP: #1922742
|
|
|
|
When network interfaces are hot-attached to the VM, attempting to get
network metadata might return 410 (or 500, 503 etc) because the info
is not yet available. In those cases, we retry getting the metadata
before giving up. The only case where we can move on to wait for more
nic attach events is if the call times out despite retries, which
means the interface is not likely a primary interface, and we should
try for more nic attach events.
|
|
This change allows us to retrieve the username and hostname from
IMDS instead of having to rely on the mounted OVF.
|
|
Due to hyper-v implementations, iso ejection is more efficient if performed
from within the guest. The code will attempt to perform a best-effort ejection.
Failure during ejection will not prevent reporting ready from happening. If iso
ejection is successful, later iso ejection from the platform will be a no-op.
In the event the iso ejection from the guest fails, iso ejection will still happen at
the platform level.
|
|
the above option allows the user to control the behavior of a distro
hostname selection if both short hostname and FQDN are supplied.
If `prefer_fqdn_over_hostname` is true the FQDN will be selected as
hostname; if false the hostname will be selected
LP: #1921004
|
|
Invoking walinuxagent from within cloud-init is no longer
supported/necessary
|
|
This PR adds in support so that cloud-init can run on instances
deployed on Vultr cloud. This was originally brought up in #628.
Co-authored-by: Eric Benner <ebenner@vultr.com>
|
|
Update sysconfig configuration to use BONDING_MODULES_OPTS instead of
BONDING_OPTS when on a SUSE system. The sysconfig support requires use
of BONDING_MODULE_OPTS whereas the initscript support that rhel uses
requires BONDING_OPTS.
|
|
This patch adds support to resize a single partition of a VM if it's using an
LVM underneath. The patch detects if it's LVM if the given block device
is a device mapper by its name (e.g. `/dev/dm-1`) and if it has slave
devices under it on sysfs. After that syspath is updated to the real
block device and growpart will be called to resize it (and automatically
its Physical Volume).
The Volume Group will be updated automatically and a final call to
extend the rootfs to the remaining space available will be made.
Using the same growpart configuration, the user can specify only one
device to be resized when using LVM and growpart, otherwise cloud-init
won't know which one should be resized and will fail.
rhbz: #1810878
LP: #1799953
Signed-off-by: Eduardo Otubo <otubo@redhat.com>
Signed-off-by: Scott Moser <smoser@brickies.net>
|
|
klibc initramfs in debian allows the 'iscsi_target_ip=' cmdline
parameter to specify an iscsi device attachment. This can
cause cloud-init to mis-detect the cmdline paramter as a
networking config.
LP: #1919188
|
|
Ensure that the Azure helper's http handler sleeps a fixed duration
between retry failure attempts. The http handler will sleep a fixed
duration between failed attempts regardless of whether the attempt
failed due to (1) request timing out or (2) instant failure (no
timeout).
Due to certain platform issues, the http request to the Azure endpoint
may instantly fail without reaching the http timeout duration. Without
sleeping a fixed duration in between retry attempts, the http handler
will loop through the max retry attempts quickly. This causes the
communication between cloud-init and the Azure platform to be less
resilient due to the short total duration if there is no sleep in
between retries.
|
|
Prior to this commit, when a user specified configuration which would
generate random passwords for users, cloud-init would cause those
passwords to be written to the serial console by emitting them on
stderr. In the default configuration, any stdout or stderr emitted by
cloud-init is also written to `/var/log/cloud-init-output.log`. This
file is world-readable, meaning that those randomly-generated passwords
were available to be read by any user with access to the system. This
presents an obvious security issue.
This commit responds to this issue in two ways:
* We address the direct issue by moving from writing the passwords to
sys.stderr to writing them directly to /dev/console (via
util.multi_log); this means that the passwords will never end up in
cloud-init-output.log
* To avoid future issues like this, we also modify the logging code so
that any files created in a log sink subprocess will only be
owner/group readable and, if it exists, will be owned by the adm
group. This results in `/var/log/cloud-init-output.log` no longer
being world-readable, meaning that if there are other parts of the
codebase that are emitting sensitive data intended for the serial
console, that data is no longer available to all users of the system.
LP: #1918303
|
|
The locale wasn't persisted correct nor set.
LP: #1402406
|
|
#342 (70dbccbb) introduced the ability to determine route-metrics based on
the `device-number` provided by the EC2 IMDS. Not all datasources that
subclass EC2 will have this attribute, so allow the old behavior if
`device-number` is not present.
LP: #1917875
|
|
`get_interfaces` is used to in two ways, broadly: firstly, to determine
the available interfaces when converting cloud network configuration
formats to cloud-init's network configuration formats; and, secondly, to
ensure that any interfaces which are specified in network configuration
are (a) available, and (b) named correctly. The first of these is
unaffected by this commit, as no clouds support Open vSwitch
configuration in their network configuration formats.
For the second, we check that MAC addresses of physical devices are
unique. In some OVS configurations, there are OVS-created devices which
have duplicate MAC addresses, either with each other or with physical
devices. As these interfaces are created by OVS, we can be confident
that (a) they will be available when appropriate, and (b) that OVS will
name them correctly. As such, this commit excludes any OVS-internal
interfaces from the set of interfaces returned by `get_interfaces`.
LP: #1912844
|
