| Age | Commit message (Collapse) | Author |
|---|
|
Fujitsu Cloud Service attaches a ovf iso transport with a label
'OVFENV'. This seems to be a reasonable value as a label.
While the for bug 1731868 would likely fix cloud-init on fujitsu
cloud, this change will find it faster.
LP: #1698669
|
|
read-version --json would report bad data when working in a worktree.
This is just because in a worktree, .git is not a directory, but
rather a metadata file that points to the another path.
$ git worktree ../mytree
$ cat ../mytree/.git
gitdir: /path/to/cloud-init/.git/worktrees/mytree
$ rm -Rf ../mytree; git worktree prune
|
|
New mkfs.vfat and fatlabel tools included in the dosfsutils package no
longer support creating vfat disks with lowercase labels. They silently
default to an all uppercase label eg CONFIG-2 instead of config-2. This
change makes cloud-init handle either upper or lower case.
LP: #1598783
|
|
The previous OVF datasource change added a debug message that referenced
an un-used variable. The failure path would be triggered if an image was
booted with a iso9660 filesystem attached to a device that was not a
cdrom.
A unit test is added for the specific failure found.
Additional safety to avoid 'cidata' labels is also added to the OVF
checker.
LP: #1737704
|
|
Previously the OVF transport would not be identified except for when
config files set 'ovf_vmware_guest_customization'. It would also
return DS_MAYBE almost always.
The change here is to add support to ds-identify for storing the
iso9660 filesystems that it finds (ISO9660_DEVS). Then the OVF check
will check that the iso9660 filesystem has ovf-env.xml on it. The least
wonderful part of this is that the check is done by 'grep' for case
insensitive ovf-env.xml.
Future improvement would be to identify VMware's OVF by label or UUID
so we could avoid the grep.
LP: #1731868
|
|
The motivation for this is that
a.) 1.7.1 runs with python 3.6 (bionic)
b.) we want to run pylint on tests/ and tools for the same reasons
that we want to run it on cloudinit/
The changes are described below.
- Update tox.ini to invoke pylint v1.7.1.
- Modify .pylintrc generated-members ignore mocked object members (m_.*)
- Replace "dangerous" params defaulting to {}
- Fix up cloud_tests use of platforms
- Cast some instance objects to with dict()
- Handle python2.7 vs 3+ ConfigParser use of readfp (deprecated)
- Update use of assertEqual(<boolean>, value) to assert<Boolean>(value)
- replace depricated assertRegexp -> assertRegex
- Remove useless test-class calls to super class
- Assign class property accessors a result and use it
- Fix missing class member in CepkoResultTests
- Fix Cheetah test import
|
|
During continuous integration tests, we're seeing quite a lot of
unreliablity when running 'yum install'. The change here is to move to
re-trying a run of 'yum install --downloadonly' for 10 times or until
it succeeds. Then afterwards, running yum install from the cache.
This seems safer in general than just re-trying an install operation,
since we are specifically affected by the download phase failing.
Also present are some flake8 fixes to tools/read-dependencies.
|
|
Per centos documentation using the fastestmirror plugin is effective at
finding the fastest mirror, unless you are behind a proxy. In that case
you should disable it. Therefore, in our tests if we are setting the proxy
we should also disable the fastestmirror plugin.
|
|
The tools that use "git describe" were just assuming a consisent
number of characters in the hash. It seems ubuntu 16.04 would use 7
and later versions use 8. To avoid that discrepency in developer
environments, set it to 8.
|
|
The first revision of this rendered tables with less decoration but there
was a desire upstream to avoid possibly breaking some parsing someone
might be doing, so it has been revised to render the same as prettytable
for the cases cloud-init actually uses.
|
|
Things done here:
- identify 'suse' as a variant in util.system_info and
also tools/render-cloudcfg.
- update systemd and cloud.cfg templates for suse specific changes.
LP: #1718640
|
|
The xkvm script will be utilized by pending NoCloud qemu testing.
If this turns out to not be the case, then we will drop it.
|
|
OpenStack Nova identifies itself only to Intel guests.
Make ds-identify return 'MAYBE' for OpenStack on non-intel arches.
An unnecessary change here is to rename the 'policy_nodmi' kwarg
to 'policy_no_dmi' in the related unit tests.
LP: #1715241
|
|
If you ran tools/run-centos without an argument it would fail due
to 'set -u' like:
./tools/run-centos: line 266: 1: unbound variable
|
|
Here we add and enable by default a datasource for Scaleway cloud.
The datasource quickly exits unless one of three things:
a.) 'Scaleway' found as the system vendor
b.) 'scaleway' found on the kernel command line.
c.) the directory /var/run/scaleway exists (this is currently created
by the scaleway initramfs module).
One interesting bit of this particular datasource is that it requires
the source port of the http request to be < 1024.
|
|
We should be expecting IndexError instead of KeyError because we are
using a list (key_ids) and not a dictionary. Also, thanks to Emmanuel
Kasper for pointing out the wrong response code.
LP: #1701527
|
|
- Simplify the logic of 'variant' in util.system_info
much of the data from
https://github.com/hpcugent/easybuild/wiki/OS_flavor_name_version
- fix get_resource_disk_on_freebsd when running on a system without
an Azure resource disk.
- fix tools/build-on-freebsd to replace oauth with oauthlib and add
bash which is a dependency for tests.
- update a fiew places that were checking for freebsd but not using
the util.is_FreeBSD()
|
|
read-dependencies now takes --test-distro param to indicate we want to install
all system package depenencies to allow for testing and building for our
continous integration environment. It allows us to install all needed deps on
a fresh system with:
python3 ./tools/read-dependencies --distro ubuntu --test-distro [--dry-run].
Additionally read-dependencies now looks at what version of python is running
the script (py2 vs p3) and opts to install python 2 or 3 system deps
respectively. This behavior can still be overridden with
python3 ./tools/read-dependencies ... --python-version 2.
There are also some distro-specific packaging and test dependencies, like
devscripts, tox and libssl-dev on debian or ubuntu. Those pkg dependencies
have now been broken out from common pkg deps to avoid trying to install them
on centos/redhat/suse.
|
|
These changes are all in an effort to get tools/run-centos using
read-dependencies rather than the 'setup-centos' script with a separate
set of dependencies listed.
- tools/read-dependencies: support taking multiple --requirements
options. This allows run-centos to get both test and build
dependencies. Ultimately, I think it might be nicer for
read-dependencies to take a list of "goals" (build, test, run or
test-tox) rather than having the caller need to know to provide
multiple --requirements.
- packages/pkg-deps.json: drop the version on the sudo package.
centos 6 has newer (1.8.6p3) version than listed, so its not a problem.
- test_handler_disk_setup.py: a test case here was using assertLogs
which is not present in the version of unittest2 that is available in
centos 6 epel. We just adjust it to use with_logs = True.
- tools/run-cents:
- improve usage with example
- add 'inside_as_cd' to provide the dir you want to cd first to.
- avoid the intermediate tarball on disk in the container.
- add 'prep' subcommand and use it to install pre-dependencies.
- use read-dependencies.
|
|
This change adds a couple of makefile targets for ci environments to
install all necessary dependencies for package builds and test runs.
It adds a number of arguments to ./tools/read-dependencies to facilitate
reading pip dependencies, translating pip deps to system package names and
optionally installing needed system-package dependencies on the local
system. This relocates all package dependency and translation logic into
./tools/read-dependencies instead of duplication found in packages/brpm
and packages/bddeb.
In this branch, we also define buildrequires as including all runtime
requires when rendering cloud-init.spec.in and debian/control files
because our package build infrastructure will also be running all unit
test during the package build process so we need runtime deps at build
time.
Additionally, this branch converts
packages/(redhat|suse)/cloud-init.spec.in from cheetah templates to jinja
to allow building python3 envs.
|
|
The added 'run-centos' does:
- Creates centos 6 or 7 lxd container
* Sets http_proxy variable for yum if set locally
* Creates centos user
- Push local tree
* Tar's up working directory
* Pushes to container and untars
- Installs pip and yum dependencies
- As user centos it can then based on flags:
* runs unittests
* run ./packages/brpm
* run ./packages/brpm --srpm
* artifact the built *.rpm
|
|
Here we move the config/cloud.cfg to be rendered as a template.
That allows us to maintain deltas between distros in one place.
Currently we use 'variant' variable to make decisions.
A tools/render-cloudcfg is provided to render the file.
There were changes to setup.py, MANIFEST.in to allow us to put all
files into a virtual env installation and to render the cloud-config
file in 'install' or 'bdist' targets.
We have also included some config changes that were found in the
redhat distro spec.
* include some config changes from the redhat distro spec.
The rendered cloud.cfg has some differences.
Ubuntu: white space and comment changes only.
Freebsd:
- whitespace changes and comment changes
- datasource_list definition moved to be closer to 'datasource'.
- enable modules: migrator, write_files
- move package-update-upgrade-install to final.
The initial work was done by Josh Harlow.
|
|
This allows the user to seed NoCloud in a trivial way from qemu/libvirt,
by using a stock image and passing a single command line flag. No custom
command line, no filesystem modification, no bootstrap disk image.
This is particularly handy now that Ec2 backend is discouraged from use
under bug 1660385.
LP: #1691772
|
|
Azure sets a known chassis asset tag to 7783-7084-3265-9085-8269-3286-77.
We can inspect this in both ds-identify and DataSource.get_data to
determine whether we are on Azure.
Added unit tests to cover these changes
and some minor tweaks to Exception error message content to give more
context on malformed or missing ovf-env.xml files.
LP: #1693939
|
|
Older cloud-init versions have a bug in the signature of the
render_network_state method for netplan (bug 1685944).
The old had:
render_network_state(target, network_state)
The fix was to change netplan's so it had the correct signature:
render_network_state(network_state, target)
This just changes our caller to use kwargs style when invoking that
method so that it works with either the broken form or correct form.
|
|
cloud-config files are very flexible and permissive.
This adds a jsonsschema definition to the cc_ntp module and validation
functions in cloudinit/config/schema which will log warnings about
invalid configuration values in the ntp section.
A cmdline tools/cloudconfig-schema is added which can be used in our dev
environments to quickly attempt to exercise the ntp schema.
It is also exposed as a main in cloudinit.config.schema.
(python3 -m cloudinit.config.schema)
LP: #1692916
|
|
AliYun cloud platform is now identifying themselves by setting the dmi
product id to the well known value "Alibaba Cloud ECS". The changes here
identify that properly in tools/ds-identify and in the DataSourceAliYun.
Since the 'get_data' for AliYun now identifies itself correctly, we can
enable AliYun by default.
LP: #1638931
|
|
If the user configured:
datasource_list: ["Ec2", "None"]
then ds-identify would write
datasource_list: ["Ec2", "None", "None"]
which would break the logic to avoid warning.
|
|
This just moves flake8 and related tools up to newer versions and fixes
the complaints associated with that.
We added to the list of flake8 ignores:
H102: do not put vim info in source files
H304: no relative imports
Also updates and pins the following in the flake8 environment:
pep8: 1.7.0 => drop (although hacking still pulls it in).
pyflakes 1.1.0 => 1.5.0
hacking 0.10.2 => 0.13.0
flake8 2.5.4 => 3.3.0
pycodestyle none => 2.3.1
|
|
This change allows the presence of
"/var/lib/cloud/instance/warnings/.skip" to
actually skip warnings as documented in Z99-cloudinit-warnings.sh.
LP: #1691551
|
|
This reverts commit 64a3df16d9c63db470a3ba55d9c5cc8e05d050d7.
Patch shouldn't have been pulled. The issue was inside the netplan renderer.
|
|
We're calling Renderer.render_network_state() with incorrect args.
% PYTHONPATH=`pwd` ./tools/net-convert.py \
--network-data=simple-v2.yaml --kind=yaml \
--output-kind netplan --directory ./target
Traceback (most recent call last):
File "./tools/net-convert.py", line 82, in <module>
main()
File "./tools/net-convert.py", line 78, in main
r.render_network_state(ns, target=args.directory)
TypeError: render_network_state() got multiple values for argument 'target'
The method signature requires passing <target dir>, <network_state>.
This patch fixes the call order.
LP: #1685944
|
|
This patch targets to make FreeBSD 10.3 or 11 work on Azure. The
modifications abide by the rule of:
* making as less modification as possible
* delegate to the distro or datasource where possible.
The main modifications are:
1. network configuration improvements, and movement into distro path.
2. Fix setting of password.
Password setting through "pw" can only work through pipe.
3. Add 'root:wheel' to syslog_fix_perms field.
4. Support resizing default file system (ufs)
5. copy cloud.cfg for freebsd to /etc/cloud/cloud.cfg rather than
/usr/local/etc/cloud/cloud.cfg.
6. Azure specific changes:
a. When reading the azure endpoint, search in a different path
and read a different option name (option-245 vs. unknown-245).
so, the lease file path should be generated according to platform.
b. adjust the handling of ephemeral mounts for ufs filesystem and
for finding the ephemeral device.
c. fix mounting of cdrom
LP: #1636345
|
|
This adds several unit tests for ds-identify, and fixes a bug
in Ec2 detection that I found while writing these tests.
The method of testing is to use the ds-identify code as a shell
library. The TestDsIdentify:call basically does:
* populate a (temp) directory with files that represent what
ds-identify would see in /sys or other locations it reads.
* create a file '_shwrap' that replaces the 3 programs that are executed
in ds-identify code path. It supports setting their stdout, stderr,
and exit code.
* set the default policies explicitly (DI_DEFAULT_POLICY) so we can
support testing different builtins. This is necessary because the
Ubuntu branches patch the builtin value. If we did not explicilty set
it, then testing there would fail.
* execute sh to source the script and call its main.
|
|
Apparently signals were crossed when this implementation was done.
Cloud-init was reading 'platform' in the environment of pid 1, but
nova-lxd was setting 'product_name'.
The fix is being made here in cloud-init to instead read product_name.
LP: #1685810
|
|
The discard action (tilde character) has been replaced by the
“stop” RainerScript directive. It is considered more intuitive and
offers slightly better performance.
The tilde operator was deprecated in rsyslog 7. Distributions
using rsyslog older than that will need to patch.
LP: #1367899
|
|
This will change all instances of LOG.warn to LOG.warning as warn
is now a deprecated method. It will also make sure any logging
uses lazy logging by passing string format arguments as function
parameters.
|
|
We were checking /var/lib/cloud/openstack/latest/meta_data.json instead
of /var/lib/cloud/seed/config_drive/openstack/latest/meta_data.json.
LP: #1673637
|
|
The reading of MAAS datasource configuration was simply broken.
it was looking in /etc/cloud/*maas*.cfg rather than
/etc/cloud/cloud.cfg.d/*maas*.cfg.
along side here there is also:
* doc improvement on check_config
* remove the path restrictions when searching for values in both
maas and ovf_vmware_guest_customization. that was done to improve
performance as check_config's parsing is slow.
* change to maas to search all config files rather than restricting
to a subset as it tried before. that was done for
* better variable names.
- rename path_cloud_confd to path_etc_cloud
- PATH_ETC_CLOUD: /etc/cloud
- PATH_ETC_CI_CFG: /etc/cloud/cloud.cfg
- PATH_ETC_CI_CFG_D: /etc/cloud/cloud.cfg.d
LP: #1677710
|
|
If the only the None datasource was listed in datasource_list, then
ds-identify would write a cloud.cfg witih:
datasource_list: [None, None]
The fix is to just append None if the list only has None.
|
|
OpenStack clouds installed with RedHat RDO have the nova product
configured in /etc/nova/release to be 'OpenStack Compute' rather than
upstream nova default of 'OpenStack Nova'.
This was first reported on Finnish provider Nebula (http://nebula.fi).
LP: #1675349
|
|
While documentation indicates that the smbios product name should
contain 'Google Compute Engine', experimentation and bug reports
indicate that is not always the case. The change here is to change
the check for GCE to also consider a serial number that starts with
'GoogleCompute-'.
Also, ds-identify was not currently searching for GCE if no config of
datasource_list was found. Most images have a datasource_list defined.
So update the list to include GCE.
LP: #1674861
|
|
This is thie cloud-init part of a fix to allow nova-lxd to provide
config drive data. The other part will be done in nova-lxd.
The agreement here is that nova-lxd will copy the contents of the
config drive to /config-drive in the container.
LP: #1673411
|
|
The path for checking presence of Bigstep datasource was simply wrong.
Set the correct path.
LP: #1674766
|
|
The script is written to have the protection of disabling filename
expansion (set -f) and explicitly enabling expansion when needed.
However, the check_config function failed to disable it after enabling.
|
|
Network configuration version 2 format is implemented in a package
called netplan (nplan)[1] which allows consolidated network config
for multiple network controllers.
- Add a new netplan renderer
- Update default policy, placing eni and sysconfig first
This requires explicit policy to enable netplan over eni
on systems which have both (Yakkety, Zesty, UC16)
- Allow any network state (parsed from any format cloud-init supports) to
render to v2 if system supports netplan.
- Move eni's _subnet_is_ipv6 to common code for use by other renderers
- Make sysconfig renderer always emit /etc/syconfig/network configuration
- Update cloud-init.service systemd unit to also wait on
systemd-networkd-wait-online.service
1. https://lists.ubuntu.com/archives/ubuntu-devel/2016-July/039464.html
|
|
Change policy so that 'report' can be overridden.
In xenial we had set the builtin default to be 'report', expecting
that Ubuntu core would install config that changed it to 'search'.
However, if report was already set, there was no way to unset it.
The change here is to make 'report' basically 'search-dryrun', so
that one or the other can be set.
The other change here is that report would actually exit disabled
if it did not find a datasource and notfound=disabled. That was
unexpected and would turn cloud-init off, which is not what we wanted.
Additionally, consistently use 'enabled' or 'disabled' versus
'enable' and 'disable'.
LP: #1669949
|
|
Rather than having the dscheck_Ec2 just know the setting, move
it up to a more formal declaration. This will make it look more
clean when a distro carries a patch to change it to warn.
|
|
On a 'not found' result, was not being written at all.
That had the unintended effect of '--force' not working.
Now, on a 'not found' result:
- if reporting: write the list as found (with just 'None').
- if not reporting: only report that there was nothing found.
this means that the warning cloud-init will write about ds-identify
failing to find a datasource will be written, but cloud-init will
still search its fully configured list.
|
|
Now, when ds-identify runs in report mode, it still writes to
/run/cloud-init.cfg as search does, but it will namespace the
result under the top level 'di_report' entry.
|
