| Age | Commit message (Collapse) | Author |
|---|
|
This patch targets to make FreeBSD 10.3 or 11 work on Azure. The
modifications abide by the rule of:
* making as less modification as possible
* delegate to the distro or datasource where possible.
The main modifications are:
1. network configuration improvements, and movement into distro path.
2. Fix setting of password.
Password setting through "pw" can only work through pipe.
3. Add 'root:wheel' to syslog_fix_perms field.
4. Support resizing default file system (ufs)
5. copy cloud.cfg for freebsd to /etc/cloud/cloud.cfg rather than
/usr/local/etc/cloud/cloud.cfg.
6. Azure specific changes:
a. When reading the azure endpoint, search in a different path
and read a different option name (option-245 vs. unknown-245).
so, the lease file path should be generated according to platform.
b. adjust the handling of ephemeral mounts for ufs filesystem and
for finding the ephemeral device.
c. fix mounting of cdrom
LP: #1636345
|
|
This adds several unit tests for ds-identify, and fixes a bug
in Ec2 detection that I found while writing these tests.
The method of testing is to use the ds-identify code as a shell
library. The TestDsIdentify:call basically does:
* populate a (temp) directory with files that represent what
ds-identify would see in /sys or other locations it reads.
* create a file '_shwrap' that replaces the 3 programs that are executed
in ds-identify code path. It supports setting their stdout, stderr,
and exit code.
* set the default policies explicitly (DI_DEFAULT_POLICY) so we can
support testing different builtins. This is necessary because the
Ubuntu branches patch the builtin value. If we did not explicilty set
it, then testing there would fail.
* execute sh to source the script and call its main.
|
|
Apparently signals were crossed when this implementation was done.
Cloud-init was reading 'platform' in the environment of pid 1, but
nova-lxd was setting 'product_name'.
The fix is being made here in cloud-init to instead read product_name.
LP: #1685810
|
|
The discard action (tilde character) has been replaced by the
“stop” RainerScript directive. It is considered more intuitive and
offers slightly better performance.
The tilde operator was deprecated in rsyslog 7. Distributions
using rsyslog older than that will need to patch.
LP: #1367899
|
|
This will change all instances of LOG.warn to LOG.warning as warn
is now a deprecated method. It will also make sure any logging
uses lazy logging by passing string format arguments as function
parameters.
|
|
We were checking /var/lib/cloud/openstack/latest/meta_data.json instead
of /var/lib/cloud/seed/config_drive/openstack/latest/meta_data.json.
LP: #1673637
|
|
The reading of MAAS datasource configuration was simply broken.
it was looking in /etc/cloud/*maas*.cfg rather than
/etc/cloud/cloud.cfg.d/*maas*.cfg.
along side here there is also:
* doc improvement on check_config
* remove the path restrictions when searching for values in both
maas and ovf_vmware_guest_customization. that was done to improve
performance as check_config's parsing is slow.
* change to maas to search all config files rather than restricting
to a subset as it tried before. that was done for
* better variable names.
- rename path_cloud_confd to path_etc_cloud
- PATH_ETC_CLOUD: /etc/cloud
- PATH_ETC_CI_CFG: /etc/cloud/cloud.cfg
- PATH_ETC_CI_CFG_D: /etc/cloud/cloud.cfg.d
LP: #1677710
|
|
If the only the None datasource was listed in datasource_list, then
ds-identify would write a cloud.cfg witih:
datasource_list: [None, None]
The fix is to just append None if the list only has None.
|
|
OpenStack clouds installed with RedHat RDO have the nova product
configured in /etc/nova/release to be 'OpenStack Compute' rather than
upstream nova default of 'OpenStack Nova'.
This was first reported on Finnish provider Nebula (http://nebula.fi).
LP: #1675349
|
|
While documentation indicates that the smbios product name should
contain 'Google Compute Engine', experimentation and bug reports
indicate that is not always the case. The change here is to change
the check for GCE to also consider a serial number that starts with
'GoogleCompute-'.
Also, ds-identify was not currently searching for GCE if no config of
datasource_list was found. Most images have a datasource_list defined.
So update the list to include GCE.
LP: #1674861
|
|
This is thie cloud-init part of a fix to allow nova-lxd to provide
config drive data. The other part will be done in nova-lxd.
The agreement here is that nova-lxd will copy the contents of the
config drive to /config-drive in the container.
LP: #1673411
|
|
The path for checking presence of Bigstep datasource was simply wrong.
Set the correct path.
LP: #1674766
|
|
The script is written to have the protection of disabling filename
expansion (set -f) and explicitly enabling expansion when needed.
However, the check_config function failed to disable it after enabling.
|
|
Network configuration version 2 format is implemented in a package
called netplan (nplan)[1] which allows consolidated network config
for multiple network controllers.
- Add a new netplan renderer
- Update default policy, placing eni and sysconfig first
This requires explicit policy to enable netplan over eni
on systems which have both (Yakkety, Zesty, UC16)
- Allow any network state (parsed from any format cloud-init supports) to
render to v2 if system supports netplan.
- Move eni's _subnet_is_ipv6 to common code for use by other renderers
- Make sysconfig renderer always emit /etc/syconfig/network configuration
- Update cloud-init.service systemd unit to also wait on
systemd-networkd-wait-online.service
1. https://lists.ubuntu.com/archives/ubuntu-devel/2016-July/039464.html
|
|
Change policy so that 'report' can be overridden.
In xenial we had set the builtin default to be 'report', expecting
that Ubuntu core would install config that changed it to 'search'.
However, if report was already set, there was no way to unset it.
The change here is to make 'report' basically 'search-dryrun', so
that one or the other can be set.
The other change here is that report would actually exit disabled
if it did not find a datasource and notfound=disabled. That was
unexpected and would turn cloud-init off, which is not what we wanted.
Additionally, consistently use 'enabled' or 'disabled' versus
'enable' and 'disable'.
LP: #1669949
|
|
Rather than having the dscheck_Ec2 just know the setting, move
it up to a more formal declaration. This will make it look more
clean when a distro carries a patch to change it to warn.
|
|
On a 'not found' result, was not being written at all.
That had the unintended effect of '--force' not working.
Now, on a 'not found' result:
- if reporting: write the list as found (with just 'None').
- if not reporting: only report that there was nothing found.
this means that the warning cloud-init will write about ds-identify
failing to find a datasource will be written, but cloud-init will
still search its fully configured list.
|
|
Now, when ds-identify runs in report mode, it still writes to
/run/cloud-init.cfg as search does, but it will namespace the
result under the top level 'di_report' entry.
|
|
Z99-cloudinit-warnings.sh can be dropped into /etc/profile.d.
Warnings that are written to /var/lib/cloud/instance/warnings
will be displayed to the user on stderr when they log in.
|
|
Modify upstream packaging to install this file, which was
already installed in ubuntu packaging.
Also, white space changes from tabs to spaces. Very few things
in cloud-init are tabs now.
Lastly, remove the executable bit on this as ait is not necessary.
Scripts in /etc/profile.d do not have executable bit.
|
|
In the interest of speed I had skipped the parsing of
/etc/cloud/cloud.cfg for the ec2 strict_id setting. In hindsight
it seems reasonable for people to put settings there.
|
|
ovf_vmware_guest_customization defaults to true in cloud-init, meaning
that such customization is disabled. We just missed a return value
causing ovf_vmware_guest_customization to effectively default to on.
Also, when looking for setting look at /etc/cloud/cloud.cfg.
This had been omitted in interest of performance, but we should
be looking there.
|
|
cloud-init by default sets 'disable_vmware_customization' to True.
So in ds-identify, we will enable the ovf datasource if:
- virt is vmware
- 'libdeployPkgPlugin.so' exists as installed by vmware-tools or
open-vm-tools.
- disable_vmware_customization is configured to True
|
|
Brightbox will identify their platform to the guest by setting the
product serial to a string that ends with 'brightbox.com'.
LP: #1661693
|
|
Based on the setting Datasource/Ec2/strict_id, the datasource
will now warn once per instance.
|
|
ds-identify will now read this setting, and thus allow the user
to modify ds-identifies behavior via either:
1. builtin setting here cloud-init/ds-identify builtin
2. ds-identify config (/etc/cloud/ds-identify.cfg)
3. system config (/etc/cloud/cloud.cfg.d/*Ec2*.cfg)
4. kernel command line (ci.datasource.ec2.strict_id=true)
|
|
A check function that returns found or maybe can also now
return config that will be written to the resultant /run/cloud.cfg.
They do so by setting the variable _RET_excfg.
|
|
This just adds checking of the Ec2 seed directory.
|
|
The following can have cause issue:
FOO="bar ; wark"
showit() {
local b=$FOO
echo $b
}
4: local: ;: bad variable name
The answer is just to use more quotes.
|
|
Just remove some examples that are no longer valid.
|
|
This makes ds-identify run only once. Previously it would run multiple
times each boot as the generator would run more than once. This is
potentially dangerous, in that running again might find more attached
disks. However that is really only a "lucky" fix if it happens to
result differently than the first run.
Additionally, we now log the uptime that we started and ended at.
|
|
has_fs_with_label regressed when refactoring to not have leading
and trailing , in DI_FS_LABELS.
LP: #1663735
|
|
The nocloud datasource specifically would look for ds=nocloud or
ds=nocloud-net (often augmented with 'seedfrom') on the kernel command line.
Fix to return DS_FOUND in that case.
LP: #1663723
|
|
Nova lxd will now put the environment variable 'platform' into
pid 1's environment to the value 'OpenStack Nova', which is the same as
you would find in kvm guests.
LP: #1661797
|
|
aarch64 does support dmi, but OpenStack does not populate guests
with this information, and there are currently bugs in qemu preventing
it from working correctly see bug #1663304 for more information.
So, for the time being, pretend as if there is no dmi data on aarch64,
which will make it enable cloud-init even when no datasources are found.
|
|
$ pycodestyle cloudinit/ tests/ tools/
tools/make-mime.py:25:5: E722 do not use bare except'
tools/mock-meta.py:252:17: E722 do not use bare except'
For tools/make-mime.py:25:5: E722 do not use bare except' the use case is
when someone runs
./make-mime.py --attach commis
instead of
./make-mime.py --attach commissaire.txt:x-commissaire-host
The split can cause a ValueError potentially if there is no:
For tools/mock-meta.py:262:17: E722 do not use bare except' the use case is
a dictionary look up occurs potentially when an unknown key is given:
key_name = key_ids[key_id]
Do note that version 2.3.0 falsely reported a dozen or so E302 and E306
errors.
|
|
ds-identify is run here from the generator. If ds-identify does
not see any datasources, it can completely disable cloud-init.
The big value in this is that if there is no datasource, no python
will ever be loaded, and cloud-init will be disabled.o
The default policy being added here is:
search,found=all,maybe=all,notfound=disabled
That means:
- enable (in 'datasource_list') all sources that are found.
- if none are found, enable all 'maybe'.
- if no maybe are found, then disable cloud-init.
On platforms without DMI (everything except for aarch64 and x86),
the default 'notfound' setting is 'enabled'. This is because many of
the detection mechanisms rely on dmi data, which is present only on
x86 and aarch64.
|
|
Fix mock-meta to work with python2 or python3.
Additionally, it will now listen to ipv6 connections, where
previously it would only work with ipv4.
|
|
running 'make' on a git branch other than master would fail with
complaint that the tools/read-version reported a different version
than the code.
Change to only consider tags starting with 0-9 in read-version.
|
|
The change here is to use '/usr/bin/env python' in validate-yaml.py
as all other tools/*.py do.
Additionally, change the Makefile to invoke validate-yaml.py with
the python that it has selected for other things (PYVER).
|
|
This has been a recurring ask and we had initially just made the change to
the cloud-init 2.0 codebase. As the current thinking is we'll just
continue to enhance the current codebase, its desirable to relicense to
match what we'd intended as part of the 2.0 plan here.
- put a brief description of license in LICENSE file
- put full license versions in LICENSE-GPLv3 and LICENSE-Apache2.0
- simplify the per-file header to reference LICENSE
- tox: ignore H102 (Apache License Header check)
Add license header to files that ship.
Reformat headers, make sure everything has vi: at end of file.
Non-shipping files do not need the copyright header,
but at the moment tests/ have it.
|
|
Python 3 would fail to load yaml from doc/examples/cloud-config-apt.txt
when the LANG (specifically LC_CTYPE) was 'C'.
The changes here do 2 things:
a.) remove the non-ascii characters from the yaml file.
b.) fix the validate-yaml.py program to decode using utf-8 specifically
rather than using the inherited settings.
This fixes it now for ascii and in the future also should non-ascii slip in.
|
|
pycodestyle 2.1.0 is in Ubuntu zesty, and complained about the
changes made here. Simple style changes. This makes 'make pep8'
pass again when built in a zesty build system with proposed enabled.
|
|
This adds some function to the generator to maintain the presense of a
flag file '/run/cloud-init/enabled' indicating that cloud-init is enabled.
Then, only run the dhclient hooks if on Azure and cloud-init is enabled.
The test for is_azure currently only checks to see that the board vendor
is Microsoft, not actually that we are on azure. Running should not be
harmful anywhere, other than slowing down dhclient.
The value of this additional code is that then dhclient having run
does not task the system with the load of cloud-init.
Additionally, some changes to config are done here.
* rename 'dhclient_leases' to 'dhclient_lease_file'
* move that to the datasource config (datasource/Azure/dhclient_lease_file)
Also, it removes the config in config/cloud.cfg that set agent_command
to __builtin__. This means that by default cloud-init still needs
the agent installed. The suggested follow-on improvement is to
use __builtin__ if there is no walinux-agent installed.
|
|
It is more efficient and cross-distribution safe to use the hooks function
from dhclient to obtain the Azure endpoint server (DHCP option 245).
This is done by providing shell scritps that are called by the hooks
infrastructure of both dhclient and NetworkManager. The hooks then
invoke 'cloud-init dhclient-hook' that maintains json data
with the dhclient options in
/run/cloud-init/dhclient.hooks/<interface>.json .
The azure helper then pulls the value from
/run/cloud-init/dhclient.hooks/<interface>.json file(s). If that file does
not exist or the value is not present, it will then fall back to the
original method of scraping the dhcp client lease file.
|
|
commit 48ec60ae changed over several tools to use X.Y.Z-XXX-gHASH
but missed tools/read-version. The end result was that
check_version failed.
|
|
Some older versions of git (Centos 6) do not have --format=tar.gz.
To work around this, create a .tar file and then compress it.
|
|
Even if there is a .git directory, we can't use git if there
is no git executable in the path. In that case just fall back
to the cloud-init version.
|
|
For upstream version directly use the output of git-describe
(X.Y.Z-number.gHASH) rather than rather than changing it to
(X.Y.Z+number.gHASH).
The rpm version does not allow '-' in Version, so we create and use
rpm_upstream_version in the rpm spec file. That is of format:
X.Y.Z+number.gHASH
|
|
Modification of the tarball became problematic, as it meant that
any tool extracting source would find the orig source tarball different.
I found this unusable when trying to use 'gbp buildpackage'.
Other changes here are to better support using python3 or python2
for the build. Makefile will try to call the right python version
and can be told which python to use.
read-version: by adding 'tiny_p' and avoiding the import of
cloudinit.util, we need less dependencies to run this.
|
