| Age | Commit message (Collapse) | Author |
|---|
|
This enables warnings produced by pylint for unused variables (W0612),
and fixes the existing errors.
|
|
Reducing timeout to 1 second as IMDS responds within a handful
of milliseconds. Also get rid of max_retries to prevent exiting
out of polling loop early due to IMDS outage / upgrade.
Reduce Azure PreProvisioning HTTP timeouts during polling to
avoid waiting an extra minute.
LP: #1752977
|
|
LP: #1754495
|
|
This change will enable azure vms to report provisioning has completed
twice, first to tell the fabric it has completed then a second time to
enable customer settings. The datasource for the second provisioning is
the Instance Metadata Service (IMDS),and the VM will poll indefinitely for
the new ovf-env.xml from IMDS.
This branch introduces EphemeralDHCPv4 which encapsulates common logic
used by both DataSourceEc2 an DataSourceAzure for temporary DHCP
interactions without side-effects.
LP: #1734991
|
|
This fixes a traceback when attempting to bounce the network after
hostname resets.
In artful and bionic ifupdown package is no longer installed in default
cloud images. As such, Azure can't use those tools to bounce the network
informing DDNS about hostname changes. This doesn't affect DDNS updates
though because systemd-networkd is now watching hostname deltas and with
default behavior to SendHostname=True over dhcp for all hostname updates
which publishes DDNS for us.
LP: #1722668
|
|
This branch resolves lints seen by pylint revision 1.8.1 and updates our
pinned tox pylint dependency used by our tox pylint target.
|
|
Each DataSource subclass must define its own get_data method. This branch
formalizes our DataSource class to require that subclasses define an
explicit dsname for sourcing cloud-config datasource configuration.
Subclasses must also override the _get_data method or a
NotImplementedError is raised.
The branch also writes /run/cloud-init/instance-data.json. This file
contains all meta-data, user-data and vendor-data and a standardized set
of metadata keys in a json blob which other utilities with root-access
could make use of. Because some meta-data or user-data is potentially
sensitive the file is only readable by root.
Generally most metadata content types should be json serializable. If
specific keys or values are not serializable, those specific values will
be base64encoded and the key path will be listed under the top-level key
'base64-encoded-keys' in instance-data.json. If json writing fails due to
other TypeErrors or UnicodeDecodeErrors, a warning log will be emitted to
/var/log/cloud-init.log and no instance-data.json will be created.
|
|
There is a race condition where our sandboxed dhclient properly writes a
lease file but has not yet written a pid file. If the sandbox temporary
directory is torn down before the dhclient subprocess writes a pidfile
DataSourceEc2Local gets a traceback and the instance will fallback to
DataSourceEc2 in the init-network stage. This wastes boot cycles we'd
rather not spend.
Fix handling of sandboxed dhclient to wait for both pidfile and leasefile
before proceding. If either file doesn't show in 5 seconds, log a warning
and return empty lease results {}.
LP: #1735331
|
|
Azure kernel now configures the SRIOV devices itself so cloud-init
does not need to provide any SRIOV device configuration or udev
naming rules.
LP: #1721579
|
|
Currently the Azure data source waits up to 60 seconds. This has proven
not to be sufficient to provide resiliency to unrelated transient failures
in other parts of the infrastructure. Azure already has logic outside of
the VM to abort hung provisioning. This changes lengthens the time out to
15 minutes.
LP: #1717611
|
|
On systems with network devices with duplicate mac addresses, cloud-init
will fail to rename the devices according to the specified network
configuration. Refactor net layer to search by device driver and device
id if available. Azure systems may have duplicate mac addresses by
design.
Update Azure datasource to run at init-local time and let Azure datasource
generate a fallback networking config to handle advanced networking
configurations.
Lastly, add a 'setup' method to the datasources that is called before
userdata/vendordata is processed but after networking is up. That is
used here on Azure to interact with the 'fabric'.
|
|
- Simplify the logic of 'variant' in util.system_info
much of the data from
https://github.com/hpcugent/easybuild/wiki/OS_flavor_name_version
- fix get_resource_disk_on_freebsd when running on a system without
an Azure resource disk.
- fix tools/build-on-freebsd to replace oauth with oauthlib and add
bash which is a dependency for tests.
- update a fiew places that were checking for freebsd but not using
the util.is_FreeBSD()
|
|
The previous commit caused test failure.
This separates out _check_freebsd_cdrom and mocks it in a test
rather than patching open.
|
|
Fix the issue caused by different commands on Linux and FreeBSD. On Linux,
we used ifdown and ifup to enable and disable a NIC, but on FreeBSD, the
counterpart is "ifconfig down" and "ifconfig up".
LP: #1697815
|
|
The current method is to attempt to mount the cdrom (/dev/cd0), if it is
successful, /dev/cd0 is configured, otherwise, it is not configured. The
problem is it forgets to check whether the mounting destination folder is
created or not. As a result, mounting attempt failed even if cdrom is
ready.
LP: #1696295
|
|
In previous commit I inadvertantly left two calls to
asset_tag = util.read_dmi_data('chassis-asset-tag')
The second did not do anything useful. Thus, remove it.
|
|
Azure sets a known chassis asset tag to 7783-7084-3265-9085-8269-3286-77.
We can inspect this in both ds-identify and DataSource.get_data to
determine whether we are on Azure.
Added unit tests to cover these changes
and some minor tweaks to Exception error message content to give more
context on malformed or missing ovf-env.xml files.
LP: #1693939
|
|
This just moves flake8 and related tools up to newer versions and fixes
the complaints associated with that.
We added to the list of flake8 ignores:
H102: do not put vim info in source files
H304: no relative imports
Also updates and pins the following in the flake8 environment:
pep8: 1.7.0 => drop (although hacking still pulls it in).
pyflakes 1.1.0 => 1.5.0
hacking 0.10.2 => 0.13.0
flake8 2.5.4 => 3.3.0
pycodestyle none => 2.3.1
|
|
Large instance types have a different disk format on the newly
partitioned ephemeral drive. So we have to adjust the logic in the
Azure datasource to recognize that a disk with 2 partitions and
an empty ntfs filesystem on the second one is acceptable.
This also adjusts the datasources's builtin fs_setup config to remove
the 'replace_fs' entry. This entry was previously ignored, and confusing.
I've clarified the doc on that also.
LP: #1686514
|
|
This patch targets to make FreeBSD 10.3 or 11 work on Azure. The
modifications abide by the rule of:
* making as less modification as possible
* delegate to the distro or datasource where possible.
The main modifications are:
1. network configuration improvements, and movement into distro path.
2. Fix setting of password.
Password setting through "pw" can only work through pipe.
3. Add 'root:wheel' to syslog_fix_perms field.
4. Support resizing default file system (ufs)
5. copy cloud.cfg for freebsd to /etc/cloud/cloud.cfg rather than
/usr/local/etc/cloud/cloud.cfg.
6. Azure specific changes:
a. When reading the azure endpoint, search in a different path
and read a different option name (option-245 vs. unknown-245).
so, the lease file path should be generated according to platform.
b. adjust the handling of ephemeral mounts for ufs filesystem and
for finding the ephemeral device.
c. fix mounting of cdrom
LP: #1636345
|
|
This will change all instances of LOG.warn to LOG.warning as warn
is now a deprecated method. It will also make sure any logging
uses lazy logging by passing string format arguments as function
parameters.
|
|
When deploying on Azure and using only cloud-init, you must "bounce" the
network interface to trigger a DDNS update. This allows dhclient to
register the hostname with Azure so that DNS works correctly on their
private networks (i.e. between vm and vm).
The agent path was already doing the bounce so this creates parity
between the built-in path and the agent.
LP: #1674685
|
|
This has been a recurring ask and we had initially just made the change to
the cloud-init 2.0 codebase. As the current thinking is we'll just
continue to enhance the current codebase, its desirable to relicense to
match what we'd intended as part of the 2.0 plan here.
- put a brief description of license in LICENSE file
- put full license versions in LICENSE-GPLv3 and LICENSE-Apache2.0
- simplify the per-file header to reference LICENSE
- tox: ignore H102 (Apache License Header check)
Add license header to files that ship.
Reformat headers, make sure everything has vi: at end of file.
Non-shipping files do not need the copyright header,
but at the moment tests/ have it.
|
|
These are just simple syntax fixes to work correctly on python2.6.
Found when testing in a centos 6 container.
|
|
Cloud-init has for some time relied on walinuxagent to do some bits
of work necessary for instance initialization. That reliance has
not been needed for a while, but we have still defaulted to it.
This change uses the "builtin" path that Daniel Watkins added
some time ago by default. Also, Adjust tests that assumed the
non-__builtin__ Azure agent_command.
LP: #1538522
|
|
This adds a call to 'activate_datasource'. That will be called
during init stage (or init-local in the event of a 'local' dsmode).
It is present so that the datasource can do platform specific operations
that may be necessary. It is passed the fully rendered cloud-config
and whether or not the instance is a new instance.
The Azure datasource uses this to address formatting of the ephemeral
devices. It does so by
a.) waiting for the device to come online
b.) removing the marker files for the disk_setup and mounts modules
if it finds that the ephemeral device has been reset.
LP: #1611074
|
|
The end result of all of these changes is to get mounts managed by
cloud-init to occur only after cloud-init.service is done. We need to do
that so that filesystems that are set up by cloud-init (in disk_setup)
do not get mounted by stale entries in /etc/fstab before
the setup occurs.
This can occur in 2 ways:
a.) new instance with old /etc/fstab
b.) same instance where disk needs adjusting (Azure resize will re-format
the ephemeral disk).
The list of changes here is:
- move mounts and disk_setup module to cloud-init.service rather
than config. cloud-init.service runs earlier in boot so it
can get those mount points done earlier.
- on systemd add 'x-systemd.requires=cloud-init.service' to fstab options
- cloud-init-local.service: add Before=basic.target
- cloud-init.service:
- extend After, Before, and Wants to multiple lines rather than one
long line.
- sort consistently with cloud-init-local.service
- add DefaultDependencies=no
- add Before=default.target
- add Conflicts=shutdown.target
LP: #1611074
|
|
This adds some function to the generator to maintain the presense of a
flag file '/run/cloud-init/enabled' indicating that cloud-init is enabled.
Then, only run the dhclient hooks if on Azure and cloud-init is enabled.
The test for is_azure currently only checks to see that the board vendor
is Microsoft, not actually that we are on azure. Running should not be
harmful anywhere, other than slowing down dhclient.
The value of this additional code is that then dhclient having run
does not task the system with the load of cloud-init.
Additionally, some changes to config are done here.
* rename 'dhclient_leases' to 'dhclient_lease_file'
* move that to the datasource config (datasource/Azure/dhclient_lease_file)
Also, it removes the config in config/cloud.cfg that set agent_command
to __builtin__. This means that by default cloud-init still needs
the agent installed. The suggested follow-on improvement is to
use __builtin__ if there is no walinux-agent installed.
|
|
It is more efficient and cross-distribution safe to use the hooks function
from dhclient to obtain the Azure endpoint server (DHCP option 245).
This is done by providing shell scritps that are called by the hooks
infrastructure of both dhclient and NetworkManager. The hooks then
invoke 'cloud-init dhclient-hook' that maintains json data
with the dhclient options in
/run/cloud-init/dhclient.hooks/<interface>.json .
The azure helper then pulls the value from
/run/cloud-init/dhclient.hooks/<interface>.json file(s). If that file does
not exist or the value is not present, it will then fall back to the
original method of scraping the dhcp client lease file.
|
|
|
|
|
|
|
|
Changing this interface to allow for easy change later.
The thing that this will enable is:
a.) maas datasource to look at the system config and see if it
is configured with the same consumer_key
b.) datasource config could allow setting a variable that it
would look at.
|
|
This adds a check in cloud-init to see if the existing (cached)
datasource is still valid. It relies on support from the Datasource
to implement 'check_instance_id'. That method should quickly determine
(if possible) if the instance id found in the datasource is still valid.
This means that we can still notice new instance ids without
depending on a network datasource on every boot.
I've also implemented check_instance_id for the superclass and for
3 classes:
DataSourceAzure (check dmi data)
DataSourceOpenstack (check dmi data)
DataSourceNocloud (check the seeded data or kernel command line)
LP: #1553815
|
|
Update make check target to run pep8 and run pyflakes or pyflakes3
depending on the value of 'PYVER'. This way the python3 build
environment does not need python2 and vice versa.
Also have make check run the 'yaml' test.
tox: have tox run pep8 in the pyflakes
|
|
|
|
This replaces the use of SharedConfig.xml in both the walinuxagent case,
and the case where we communicate with the Azure fabric ourselves.
|
|
Azure has or will be offering shortly the ability to directly define the SSH
key value instead of a fingerprint in the ovf-env.xml file. This patch
favors defined SSH keys over the fingerprint method (LP: #1506244).
|
|
Azure has or will be offering shortly the ability to directly define the SSH
key value instead of a fingerprint in the ovf-env.xml file. This patch
favors defined SSH keys over the fingerprint method (LP: #1506244).
|
|
Azure's ephemeral disks are not guaranteed to be assigned the same name by
the kernel every boot. This causes problems on ~2% of Azure instances, and
can be fixed by using udev rules to give us a deterministic path to mount;
this patch introduces those udev rules and modifies the Azure data source
to use them.
Changes to a couple of config modules were also required. In some places,
they just needed to learn to dereference symlinks. In cc_mounts this
wasn't sufficient because the dereferenced device would have been put in
/etc/fstab (rather defeating the point of using the udev rules in the
first place). A fairly hefty refactor was required to separate "is this a
valid block device?" from "what shall I put in fstab?".
LP: #1411582
|
|
|
|
The ephemeral disk will not necessarily be assigned the same name at
each boot (LP: #1411582), so we use some udev rules to ensure we always
get the right one.
|
|
The fabric provides the user password in plain text via the CDROM,
and cloud-init has previously wrote the ovf-env.xml in /var/lib/waagent
with the password in plain text. This change redacts the password.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
