| Age | Commit message (Collapse) | Author |
|---|
|
With a large size ISO file attached to iso dev, ds-identify might
grep it entirely if iso dev is ISO9660, it takes very long time to
start OS.
Resolve this by:
- Adding a checker to read the ISO size (from sysfs). If
the size of the ISO filesystem is > 10MiB then the ISO will be
ignored (logged as oversized).
- Move the ovf vmware guest customization checker to be
ahead of cdrom ovf checker, so no need check the ISO size if vmware
guest customization is enabled.
LP: #1806701
|
|
- use python3 by default
- ability to use any Python version through the PYTHON env-var
- indent with 4 spaces
- use 'set -eux'
- remove trailing whitespace
- drop the cheetah dep, Jinja2 is enough
|
|
When OpenStack is deployed on some hypervisors (such as VMware
vSphere), cloud-init doesn't detect that it needs to probe the
metadata service because the DMI product name field can't be set
to a field that is recognized by cloud-init. However, the asset
tag field can be set via flavor extra specs or image metadata.
A similar approach is already used to identify Open Telekom Cloud.
This patch allows cloud init to recognize "OpenStack Nova" or
"OpenStack Compute" in the asset tag field as an indication that
the instance being configured is running on an OpenStack platform.
LP: #1669875
|
|
In this way only the 'baseurl' mirror is used, which is easier
to allow through firewalls and proxies.
|
|
config/cloud.cfg.tmpl uses 'freebsd', not 'bsd' to identify FreeBSD.
|
|
cc_set_passwords.py depends on chpasswd binary.
|
|
When the cloned branch was not the canonical upstream and tags were not
available, tox would fail because tools/read-version would fail, and
tragically never print the advice that is in tools/read-version about
how to fix it.
This changes tools/read-version to catch the exception that is elsewhere
explicitly thrown and treat that too as an error it can handle.
|
|
NoCloud data source now accepts both 'cidata' and 'CIDATA'
as filesystem labels. This is similar to DataSourceConfigDrive's
support for 'config-2' and 'CONFIG-2'.
|
|
cloud-init-per is run via /bin/sh which requires POSIX shell
compliance and does not implement string substitution like
bash. Replace these calls with use of sed.
LP: #1819222
|
|
|
|
When using a proxy it is often useful to know in advance which mirrors
are to be contacted, so a whitelist can be set up. This is not easy when
using the yum.conf(5) mirrorlist option, as the retrieved list of mirrors
may change. The repository definition may also specify a canonical mirror
with the 'baseurl' option; this option is often commented out by default
to favor the usage of worldwide mirrors. This patch uncomments 'baseurl'
when an http_proxy is being used, so the canonical mirror is used *in
addition to* the mirrors retrieved from the mirrorlist.
|
|
ovf_vmware_transport_guestinfo is not currently tested.
It used '$1' instead of '$out' when checking for xml content in
the output of vmware-rpctool.
|
|
This adds support for reading OVF information over the
'com.vmware.guestInfo' tranport. The current implementation requires
vmware-rpctool be installed in the system.
LP: #1807466
|
|
When deploying an OVA, at least some versions of vmware
attach a cdrom with an ISO9660 filesystem label of 'OVF ENV'.
This was seen on Vmware vCenter Server, 6.0.0, 2776510.
In order to accomplish this we had to change the content of
the DI_ISO9660_DEVS variable to be comma delimited rather
than space delimited.
|
|
This update to tox-venv allows you to do:
./tools/tox-venv py3 - tests/unittests/test_util.py
|
|
There was a typo in the doc string at the top of ds-identify
(disable -> disabled). That is fixed here as well as adding some
better examples on content in /etc/cloud/ds-identify.cfg.
|
|
The error message when read-vesion is not very useful and does not help
the end-user know how to overcome the issue. This adds a short message
explaining that the user does not have the latest upstream tags and how
to get those tags.
|
|
This adds a Oracle specific datasource that functions with OCI.
It is a simplified version of the OpenStack metadata server
with support for vendor-data.
It does not support the OCI-C (classic) platform.
Also here is a move of BrokenMetadata to common 'sources'
as this was the third occurrence of that class.
|
|
Move the tools/net-convert.py to be exposed as part of 'cloud-init devel'
subcommands.
It can now be called like:
$ cloud-init devel net-convert
Or, if you just have checked out source (and no cli executable):
$ python3 -m cloudinit.cmd.devel.net_convert
or
$ python3 -m cloudinit.cmd.main devel net-convert
|
|
Bash and most other "bourne-like" shells allow declaring function
local variables via 'local'. ksh does not. Instead of using 'local'
always, use 'typeset' when the KSH_VERSION variable is present in
environment.
LP: #1784713
|
|
In order to see some of the WARNING messages added by bug 1774666
I wanted logging output of tools/net-convert. This does:
a.) add '--debug' and make it print the network state and read yaml only
if --debug is provided.
b.) set up basic logging so warnings goes to console by default and
debug goes to console if --debug is provided.
|
|
If run-container was called without --package or --binary-package, then
it would still try to copy out artifacts and would fail doing so as
there were no artifacts to collect.
Also fix a bug when only --source-package without --package.
|
|
tools/run-container is like tools/run-centos, but currently supports
the following images from lxc-images
opensuse/42.3
centos/6
centos/7
ubuntu/16.04
debian/10
debian/sid
Also here is to make installation via zypper in tools/read-dependencies
not prompt user.
|
|
SuSE builds were not getting a PATH set in generator's environment.
This may seem like mis-configuration on the system, but caused ds-identify
to fail to find blkid (or any other program).
The change here just ensures that we get /sbin /usr/sbin /bin /usr/bin
into the PATH when main is run.
LP: #1771382
|
|
In playing with a SmartOS container I found that ds-identify did
not identify the container there as a container. Systemd-detect-virt
identifies it as 'container-other'.
Also here are tests for ds-identify for the SmartOS platform
identification, and some indentation fixes in ds-identify.
|
|
We had two calls to is_ds_enabled, and the debug message looked
something like this:
is_ds_enabled returned 1: ConfigDrive NoCloud
Now instead we have just one call, and the debug message like:
is_ds_enabled(IBMCloud) = true
|
|
This fixes warnings reported by shellcheck at 0.4.6.
The complaints that we are ignoring globally (top of the file) are:
2015: Note that A && B || C is not if-then-else. C may run if A is true.
2039: In POSIX sh, 'local' is undefined.
2162: read without -r will mangle backslashes.
2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.
Most of the complaints were just noise, but a few unused variables
were reported and fixed.
Related shellcheck issues opened:
- https://github.com/koalaman/shellcheck/issues/1191
- https://github.com/koalaman/shellcheck/issues/1192
- https://github.com/koalaman/shellcheck/issues/1193
- https://github.com/koalaman/shellcheck/issues/1194
|
|
Ubuntu images on IBMCloud for 16.04 have some seed data in
/var/lib/cloud/data/seed/nocloud-net. In order to have systems with
IBMCloud enabled, we modified ds-identify detection to skip that seed
if the system was on IBMCloud. That change did not consider the
fact that IBMCloud might not be in the datasource list.
There was similar logic in the ConfigDrive datasource in ds-identify
and the datasource itself.
Config drive is now updated to only check and avoid IBMCloud if IBMCloud
is enabled. The check in ds-identify for nocloud was dropped. If a
user provides a nocloud seed on IBMCloud, then that can be used.
This means that systems running Xenial will continue to get their
old datasources.
LP: #1766401
|
|
When images are deployed from template in a production environment
the artifacts of the provisioning stage (provisioningConfiguration.cfg)
that cloud-init referenced are cleaned up. However, when provisioned
in "debug" mode (internal to IBM) the artifacts are left.
This changes the 'is_ibm_provisioning' implementations in both
ds-identify and in the IBM datasource to identify the provisioning
stage more correctly. The change is to consider provisioning only
if the provisioing file existed and there was no log file or
the log file was older than this boot.
LP: #1767166
|
|
This tool is used to assist during the creation of ubuntu packages for
release testing. Address the following on the command-line:
* --help option now print usage
* Add --orig-tarball which creates named output file
cloud-init_<release-version>.orig.tar.gz
* drop unused --verbose option
|
|
This adds a specific IBM Cloud datasource.
IBM Cloud is identified by:
a.) running on xen
b.) one of a LABEL=METADATA disk or a LABEL=config-2 disk with
UUID=9796-932E
The datasource contains its own config-drive reader that reads
only the currently supported portion of config-drive needed for
ibm cloud.
During the provisioning boot, cloud-init is disabled.
See the docstring in DataSourceIBMCloud.py for more more information.
|
|
Ubuntu 16.04 (xenial) does not have jsonschema installed by default. As
it is listed in requirements, the tox environment will always have it
installed.
Add the helper tools/pipremove that removes pip packages. Then use that
to remove jsonschema without noise of always running and ignoring a
'pip uninstall jsonschema'.
|
|
Open Telekom Cloud gen1 (Xen) hosts do not provide nova product
names in DMI but Xen HVM domU. They can however be safely identified
by the OpenTelekomCloud Chassis asset tag. OpenTelekomCloud does
use the network OpenStack DataSource, so we better detect it.
LP: #1756471
|
|
The Hetzner Cloud metadata service is an AWS-style service available
over HTTP via the link local address 169.254.169.254.
https://hetzner.com/cloud
https://docs.hetzner.cloud/
|
|
On few 64-bit platforms, the open-vm-tools package is installed at
/usr/lib64/. The DataSourceOVF is changed to search look there for the
'customization plugin'
|
|
This fixes a bug in parsing of 'blkid -o export' output. The result
of the bug meant that DI_ISO9660_DEVS did not get set correctly and
is_cdrom_ovf would not identify devices in most cases.
The tests are improved to demonstrate both multiple iso devices
and also a cdrom that doesn't sort "last" in blkid output.
The code change is to use DEVNAME as the record separator when
parsing blkid -o export rather than relying on being able to read
the empty line.
LP: #1749980
|
|
Ubuntu core seeds information to nocloud via a bind-mount of
/writable/system-data/var/lib/cloud over /var/lib/cloud.
When ds-identify runs as a systemd generator that mount is not
guaranteed to have been done. It is guaranteed at
cloud-init-local.service time, but not generator time.
Images built with 'ubuntu-image --cloud-init=user-data-file'
would have cloud-init disabled.
The fix here is just to consider the seed dir under /writable/system-data.
LP: #1747070
|
|
When we moved some tests to live under cloudinit/ we inadvertantly
failed to change all things that would run nose to include that
directory.
This changes all the 'nose' invocations to consistently run with
tests/unittests and cloudinit/.
Also, it works around, more correctly this time, a python2.6-ism with
the following code:
with assertRaises(SystemExit) as cm:
sys.exit(2)
|
|
This changes tools/run-centos to collect up your git working directory
via 'git' commands rather than just collecting the whole directory.
The reason for this is that even a clean tree that has had tox run
on it might have up to 400M of data in it.
It adds a '--dirty' flag to run-centos to collect up local changes.
|
|
Fujitsu Cloud Service attaches a ovf iso transport with a label
'OVFENV'. This seems to be a reasonable value as a label.
While the for bug 1731868 would likely fix cloud-init on fujitsu
cloud, this change will find it faster.
LP: #1698669
|
|
read-version --json would report bad data when working in a worktree.
This is just because in a worktree, .git is not a directory, but
rather a metadata file that points to the another path.
$ git worktree ../mytree
$ cat ../mytree/.git
gitdir: /path/to/cloud-init/.git/worktrees/mytree
$ rm -Rf ../mytree; git worktree prune
|
|
New mkfs.vfat and fatlabel tools included in the dosfsutils package no
longer support creating vfat disks with lowercase labels. They silently
default to an all uppercase label eg CONFIG-2 instead of config-2. This
change makes cloud-init handle either upper or lower case.
LP: #1598783
|
|
The previous OVF datasource change added a debug message that referenced
an un-used variable. The failure path would be triggered if an image was
booted with a iso9660 filesystem attached to a device that was not a
cdrom.
A unit test is added for the specific failure found.
Additional safety to avoid 'cidata' labels is also added to the OVF
checker.
LP: #1737704
|
|
Previously the OVF transport would not be identified except for when
config files set 'ovf_vmware_guest_customization'. It would also
return DS_MAYBE almost always.
The change here is to add support to ds-identify for storing the
iso9660 filesystems that it finds (ISO9660_DEVS). Then the OVF check
will check that the iso9660 filesystem has ovf-env.xml on it. The least
wonderful part of this is that the check is done by 'grep' for case
insensitive ovf-env.xml.
Future improvement would be to identify VMware's OVF by label or UUID
so we could avoid the grep.
LP: #1731868
|
|
The motivation for this is that
a.) 1.7.1 runs with python 3.6 (bionic)
b.) we want to run pylint on tests/ and tools for the same reasons
that we want to run it on cloudinit/
The changes are described below.
- Update tox.ini to invoke pylint v1.7.1.
- Modify .pylintrc generated-members ignore mocked object members (m_.*)
- Replace "dangerous" params defaulting to {}
- Fix up cloud_tests use of platforms
- Cast some instance objects to with dict()
- Handle python2.7 vs 3+ ConfigParser use of readfp (deprecated)
- Update use of assertEqual(<boolean>, value) to assert<Boolean>(value)
- replace depricated assertRegexp -> assertRegex
- Remove useless test-class calls to super class
- Assign class property accessors a result and use it
- Fix missing class member in CepkoResultTests
- Fix Cheetah test import
|
|
During continuous integration tests, we're seeing quite a lot of
unreliablity when running 'yum install'. The change here is to move to
re-trying a run of 'yum install --downloadonly' for 10 times or until
it succeeds. Then afterwards, running yum install from the cache.
This seems safer in general than just re-trying an install operation,
since we are specifically affected by the download phase failing.
Also present are some flake8 fixes to tools/read-dependencies.
|
|
Per centos documentation using the fastestmirror plugin is effective at
finding the fastest mirror, unless you are behind a proxy. In that case
you should disable it. Therefore, in our tests if we are setting the proxy
we should also disable the fastestmirror plugin.
|
|
The tools that use "git describe" were just assuming a consisent
number of characters in the hash. It seems ubuntu 16.04 would use 7
and later versions use 8. To avoid that discrepency in developer
environments, set it to 8.
|
|
The first revision of this rendered tables with less decoration but there
was a desire upstream to avoid possibly breaking some parsing someone
might be doing, so it has been revised to render the same as prettytable
for the cases cloud-init actually uses.
|
|
Things done here:
- identify 'suse' as a variant in util.system_info and
also tools/render-cloudcfg.
- update systemd and cloud.cfg templates for suse specific changes.
LP: #1718640
|
