| Age | Commit message (Collapse) | Author |
|---|
|
Applied Black and isort, fixed any linting issues, updated tox.ini
and CI.
|
|
CloudStack DNS resolution should be done against
the DNS search domain (with the final dot, DNS
resolution does not work with e.g. Fedora 34)
LP: #1942232
|
|
This was painful, but it finishes a TODO from cloudinit/subp.py.
It moves the following from util to subp:
ProcessExecutionError
subp
which
target_path
I moved subp_blob_in_tempfile into cc_chef, which is its only caller.
That saved us from having to deal with it using write_file
and temp_utils from subp (which does not import any cloudinit things now).
It is arguable that 'target_path' could be moved to a 'path_utils' or
something, but in order to use it from subp and also from utils,
we had to get it out of utils.
|
|
* ec2: Add support for AWS IMDS v2 (session-oriented)
AWS now supports a new version of fetching Instance Metadata[1].
Update cloud-init's ec2 utility functions and update ec2 derived
datasources accordingly. For DataSourceEc2 (versus ec2-look-alikes)
cloud-init will issue the PUT request to obtain an API token for
the maximum lifetime and then all subsequent interactions with the
IMDS will include the token in the header.
If the API token endpoint is unreachable on Ec2 platform, log a
warning and fallback to using IMDS v1 and which does not use
session tokens when communicating with the Instance metadata
service.
We handle read errors, typically seen if the IMDS is beyond one
etwork hop (IMDSv2 responses have a ttl=1), by setting the api token
to a disabled value and then using IMDSv1 paths.
To support token-based headers, ec2_utils functions were updated
to support custom headers_cb and exception_cb callback functions
so Ec2 could store, or refresh API tokens in the event of token
becoming stale.
[1] https://docs.aws.amazon.com/AWSEC2/latest/ \
UserGuide/ec2-instance-metadata.html \
#instance-metadata-v2-how-it-works
|
|
|
|
|
|
Network has not yet been configured in the init-local stage so the
openstack datasource will use dhcp-client to temporarily obtain an ipv4
address and query the metadata service at http://169.254.169.254 to get
network_data.json configuration. If present, the datasource will return
network_config version 1 config based on that network_data.json content.
Previously OpenStack datasource only setup dhcp on the fallback interface
so this represents a change in behavior to react to the full config
provided by openstack.
Also significant to OpenStack is the separation of a _crawl_data operation
from get_data(). crawl_data walks the available metadata services and
returns a dict of discovered content. get_data consumes the crawled_data,
caches it in the datasource and reacts to that data.
/run/cloud-init/instance-data.json now published network_data.json or
ec2_metadata key if that data is present on any datasource.
The main reasons for the separation of crawl from get_data:
* Enable performance metrics of cloud-init's metadata crawls on each
* Enable cloud-init modules and scripts to query and consume metadata
content which may have updated/changed after cloud-init's initial cache
during instance boot. (Think hotplug)
Also generalize common logic to base DataSource class/module:
* Move to a common UNSET variable up into base datasource module fix EC2,
ConfigDrive, OpenStack, SmartOS to use the global.
* Drop get_url_settings from Ec2, CloudStack and OpenStack and generalize
DataSource.get_url_params(). Allow subclasses to override url_max_wait,
url_timeout and url_retries params.
* Rename get_network_metadata bool to perform_dhcp_setup as it designates
whether EphemeralDHCPv4 setup is required before crawling metadata.
LP: #1749717
|
|
Each DataSource subclass must define its own get_data method. This branch
formalizes our DataSource class to require that subclasses define an
explicit dsname for sourcing cloud-config datasource configuration.
Subclasses must also override the _get_data method or a
NotImplementedError is raised.
The branch also writes /run/cloud-init/instance-data.json. This file
contains all meta-data, user-data and vendor-data and a standardized set
of metadata keys in a json blob which other utilities with root-access
could make use of. Because some meta-data or user-data is potentially
sensitive the file is only readable by root.
Generally most metadata content types should be json serializable. If
specific keys or values are not serializable, those specific values will
be base64encoded and the key path will be listed under the top-level key
'base64-encoded-keys' in instance-data.json. If json writing fails due to
other TypeErrors or UnicodeDecodeErrors, a warning log will be emitted to
/var/log/cloud-init.log and no instance-data.json will be created.
|
|
Systems that used systemd-networkd's dhcp client would not be able to get
information on the Azure endpoint (placed in Option 245) or the CloudStack
server (in 'server_address').
The change here supports reading these files in /run/systemd/netif/leases.
The files declare that "This is private data. Do not parse.", but at this
point we do not have another option.
LP: #1718029
|
|
A regression in 'get_latest_lease' made it ignore files starting with
'dhclient-' rather than just 'dhclient.'. The fix here is to allow those
files to be considered.
There is a lot more we could do here to better ensure that we pick the
most recent lease, but this change fixes the regression.
LP: #1717147
|
|
To query the metadata, the Cloudstack source currently scans a predefined
DHCP lease directories to find the IP of the DHCP server. This list does
not include "/var/lib/NetworkManager/" which is the default directory in
CentOS7. Add that directory to the list.
|
|
Several references that were using URL shorteners are now broken due to
their service going away, making it painful to even figure out what they
were supposed to be pointing at. Put back long URLS using '# noqa' to
make flake8 happy.
LP: #1669727
|
|
This has been a recurring ask and we had initially just made the change to
the cloud-init 2.0 codebase. As the current thinking is we'll just
continue to enhance the current codebase, its desirable to relicense to
match what we'd intended as part of the 2.0 plan here.
- put a brief description of license in LICENSE file
- put full license versions in LICENSE-GPLv3 and LICENSE-Apache2.0
- simplify the per-file header to reference LICENSE
- tox: ignore H102 (Apache License Header check)
Add license header to files that ship.
Reformat headers, make sure everything has vi: at end of file.
Non-shipping files do not need the copyright header,
but at the moment tests/ have it.
|
|
|
|
It could be that there are also 'dhclient6.leases' files in /var/lib/dhcp
when DHCPv6 is used next to DHCPv4.
This patch makes sure we only read from DHCPv4 lease files
|
|
if the Datasource does not have an entry in config, then
set it to be a empty dictionary rather than None.
Also remove places that did this elsewhere.
|
|
|
|
Different versions of the CloudStack password server respond
differently; wget handles these nicely for us, so it's easier to just
use wget.
LP: #1440263, #1464253
|
|
|
|
|
|
|
|
|
|
|
|
There might be some CloudStack deployments without the :8080 password
server, and there's no reason the rest of the data source can't be used
for them.
|
|
|
|
|
|
|
|
This seems cleaner, to avoid duplicate '/' being added.
|
|
|
|
|
|
|
|
Changes in revision 753 broke cloud-init on ubuntu, as it has a different
dhclient directory than Fedora where the change was developed and tested.
This change does 2 things:
* searches multiple directories (including /var/lib/dhcp) for the lease
files.
* adds a fallback to the old code path of choosing the default route
as the virtual router if there were no virtual routers found in the
lease files.
LP: #1089989
|
|
In CloudStack's documentation, it is stated that meta/user-data can be
retrieved from CloudStack's Virtual Router [1].
However, cloud-init retrieves these information from default gateway. VR
and default gateway may be the same machine (i.e. have the same address)
in some cases, but that is not be always true (actually, in my case, it is
not).
This change searches the lease files in /var/lib/dhclient to pick out
the dhcp-server-identifier.
It admittedly does make this specific to dhclient.
--
[1] http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0.0-incubating/html/Admin_Guide/user-data-and-meta-data.html).
LP: #1089989
|
|
|
|
are used that lazily load the metadata from the
ec2 metadata service.
1. Add a ec2_utils module that checks which version
of boto is being used and under the right versions
the metadata dictionary will be expanded.
2. Use this new ec2_utils module in the cloudstack and ec2
datasources as there entrypoints into boto.
|
|
|
|
make pep8 now is silent on precise's pep8 ( 0.6.1-2ubuntu2).
|
|
There are several changes here.
* Datasource now has a 'availability_zone' getter.
* get_package_mirror_info
* Datasource convenience 'get_package_mirror_info' that calls
the configured distro, and passes it the availability-zone
* distro has a get_package_mirror_info method
* get_package_mirror_info returns a dict that of name:mirror
this is to facilitate use of 'security' and 'primary' archive.
* this supports searching based on templates. Any template
that references undefined values is skipped. These templates
can contain 'availability_zone' (LP: #1037727)
* distro's mirrors can be arch specific (LP: #1028501)
* rename_apt_lists supports the "mirror_info" rather than single mirror
* generate_sources_list supports mirror_info, and as a result, the
ubuntu mirrors reference '$security' rather than security (LP: #1006963)
* remove the DataSourceEc2 specific mirror selection, but instead
rely on the above filtering, and the fact that 'ec2_region' is only
defined if the availability_zone looks like a ec2 az.
|
|
2. Adjust comment on sources list from depends
3. For the /etc/timezone 'writing', add a header that says created by cloud-init
|
|
|
|
1. This is very useful for testing (until we have boto timeout in a more sane manner)
|
|
|
|
|
|
objects/logging added...
|
|
|
