| Age | Commit message (Collapse) | Author |
|---|
|
If you ran tools/run-centos without an argument it would fail due
to 'set -u' like:
./tools/run-centos: line 266: 1: unbound variable
|
|
Here we add and enable by default a datasource for Scaleway cloud.
The datasource quickly exits unless one of three things:
a.) 'Scaleway' found as the system vendor
b.) 'scaleway' found on the kernel command line.
c.) the directory /var/run/scaleway exists (this is currently created
by the scaleway initramfs module).
One interesting bit of this particular datasource is that it requires
the source port of the http request to be < 1024.
|
|
We should be expecting IndexError instead of KeyError because we are
using a list (key_ids) and not a dictionary. Also, thanks to Emmanuel
Kasper for pointing out the wrong response code.
LP: #1701527
|
|
- Simplify the logic of 'variant' in util.system_info
much of the data from
https://github.com/hpcugent/easybuild/wiki/OS_flavor_name_version
- fix get_resource_disk_on_freebsd when running on a system without
an Azure resource disk.
- fix tools/build-on-freebsd to replace oauth with oauthlib and add
bash which is a dependency for tests.
- update a fiew places that were checking for freebsd but not using
the util.is_FreeBSD()
|
|
read-dependencies now takes --test-distro param to indicate we want to install
all system package depenencies to allow for testing and building for our
continous integration environment. It allows us to install all needed deps on
a fresh system with:
python3 ./tools/read-dependencies --distro ubuntu --test-distro [--dry-run].
Additionally read-dependencies now looks at what version of python is running
the script (py2 vs p3) and opts to install python 2 or 3 system deps
respectively. This behavior can still be overridden with
python3 ./tools/read-dependencies ... --python-version 2.
There are also some distro-specific packaging and test dependencies, like
devscripts, tox and libssl-dev on debian or ubuntu. Those pkg dependencies
have now been broken out from common pkg deps to avoid trying to install them
on centos/redhat/suse.
|
|
These changes are all in an effort to get tools/run-centos using
read-dependencies rather than the 'setup-centos' script with a separate
set of dependencies listed.
- tools/read-dependencies: support taking multiple --requirements
options. This allows run-centos to get both test and build
dependencies. Ultimately, I think it might be nicer for
read-dependencies to take a list of "goals" (build, test, run or
test-tox) rather than having the caller need to know to provide
multiple --requirements.
- packages/pkg-deps.json: drop the version on the sudo package.
centos 6 has newer (1.8.6p3) version than listed, so its not a problem.
- test_handler_disk_setup.py: a test case here was using assertLogs
which is not present in the version of unittest2 that is available in
centos 6 epel. We just adjust it to use with_logs = True.
- tools/run-cents:
- improve usage with example
- add 'inside_as_cd' to provide the dir you want to cd first to.
- avoid the intermediate tarball on disk in the container.
- add 'prep' subcommand and use it to install pre-dependencies.
- use read-dependencies.
|
|
This change adds a couple of makefile targets for ci environments to
install all necessary dependencies for package builds and test runs.
It adds a number of arguments to ./tools/read-dependencies to facilitate
reading pip dependencies, translating pip deps to system package names and
optionally installing needed system-package dependencies on the local
system. This relocates all package dependency and translation logic into
./tools/read-dependencies instead of duplication found in packages/brpm
and packages/bddeb.
In this branch, we also define buildrequires as including all runtime
requires when rendering cloud-init.spec.in and debian/control files
because our package build infrastructure will also be running all unit
test during the package build process so we need runtime deps at build
time.
Additionally, this branch converts
packages/(redhat|suse)/cloud-init.spec.in from cheetah templates to jinja
to allow building python3 envs.
|
|
The added 'run-centos' does:
- Creates centos 6 or 7 lxd container
* Sets http_proxy variable for yum if set locally
* Creates centos user
- Push local tree
* Tar's up working directory
* Pushes to container and untars
- Installs pip and yum dependencies
- As user centos it can then based on flags:
* runs unittests
* run ./packages/brpm
* run ./packages/brpm --srpm
* artifact the built *.rpm
|
|
Here we move the config/cloud.cfg to be rendered as a template.
That allows us to maintain deltas between distros in one place.
Currently we use 'variant' variable to make decisions.
A tools/render-cloudcfg is provided to render the file.
There were changes to setup.py, MANIFEST.in to allow us to put all
files into a virtual env installation and to render the cloud-config
file in 'install' or 'bdist' targets.
We have also included some config changes that were found in the
redhat distro spec.
* include some config changes from the redhat distro spec.
The rendered cloud.cfg has some differences.
Ubuntu: white space and comment changes only.
Freebsd:
- whitespace changes and comment changes
- datasource_list definition moved to be closer to 'datasource'.
- enable modules: migrator, write_files
- move package-update-upgrade-install to final.
The initial work was done by Josh Harlow.
|
|
This allows the user to seed NoCloud in a trivial way from qemu/libvirt,
by using a stock image and passing a single command line flag. No custom
command line, no filesystem modification, no bootstrap disk image.
This is particularly handy now that Ec2 backend is discouraged from use
under bug 1660385.
LP: #1691772
|
|
Azure sets a known chassis asset tag to 7783-7084-3265-9085-8269-3286-77.
We can inspect this in both ds-identify and DataSource.get_data to
determine whether we are on Azure.
Added unit tests to cover these changes
and some minor tweaks to Exception error message content to give more
context on malformed or missing ovf-env.xml files.
LP: #1693939
|
|
Older cloud-init versions have a bug in the signature of the
render_network_state method for netplan (bug 1685944).
The old had:
render_network_state(target, network_state)
The fix was to change netplan's so it had the correct signature:
render_network_state(network_state, target)
This just changes our caller to use kwargs style when invoking that
method so that it works with either the broken form or correct form.
|
|
cloud-config files are very flexible and permissive.
This adds a jsonsschema definition to the cc_ntp module and validation
functions in cloudinit/config/schema which will log warnings about
invalid configuration values in the ntp section.
A cmdline tools/cloudconfig-schema is added which can be used in our dev
environments to quickly attempt to exercise the ntp schema.
It is also exposed as a main in cloudinit.config.schema.
(python3 -m cloudinit.config.schema)
LP: #1692916
|
|
AliYun cloud platform is now identifying themselves by setting the dmi
product id to the well known value "Alibaba Cloud ECS". The changes here
identify that properly in tools/ds-identify and in the DataSourceAliYun.
Since the 'get_data' for AliYun now identifies itself correctly, we can
enable AliYun by default.
LP: #1638931
|
|
If the user configured:
datasource_list: ["Ec2", "None"]
then ds-identify would write
datasource_list: ["Ec2", "None", "None"]
which would break the logic to avoid warning.
|
|
This just moves flake8 and related tools up to newer versions and fixes
the complaints associated with that.
We added to the list of flake8 ignores:
H102: do not put vim info in source files
H304: no relative imports
Also updates and pins the following in the flake8 environment:
pep8: 1.7.0 => drop (although hacking still pulls it in).
pyflakes 1.1.0 => 1.5.0
hacking 0.10.2 => 0.13.0
flake8 2.5.4 => 3.3.0
pycodestyle none => 2.3.1
|
|
This change allows the presence of
"/var/lib/cloud/instance/warnings/.skip" to
actually skip warnings as documented in Z99-cloudinit-warnings.sh.
LP: #1691551
|
|
This reverts commit 64a3df16d9c63db470a3ba55d9c5cc8e05d050d7.
Patch shouldn't have been pulled. The issue was inside the netplan renderer.
|
|
We're calling Renderer.render_network_state() with incorrect args.
% PYTHONPATH=`pwd` ./tools/net-convert.py \
--network-data=simple-v2.yaml --kind=yaml \
--output-kind netplan --directory ./target
Traceback (most recent call last):
File "./tools/net-convert.py", line 82, in <module>
main()
File "./tools/net-convert.py", line 78, in main
r.render_network_state(ns, target=args.directory)
TypeError: render_network_state() got multiple values for argument 'target'
The method signature requires passing <target dir>, <network_state>.
This patch fixes the call order.
LP: #1685944
|
|
This patch targets to make FreeBSD 10.3 or 11 work on Azure. The
modifications abide by the rule of:
* making as less modification as possible
* delegate to the distro or datasource where possible.
The main modifications are:
1. network configuration improvements, and movement into distro path.
2. Fix setting of password.
Password setting through "pw" can only work through pipe.
3. Add 'root:wheel' to syslog_fix_perms field.
4. Support resizing default file system (ufs)
5. copy cloud.cfg for freebsd to /etc/cloud/cloud.cfg rather than
/usr/local/etc/cloud/cloud.cfg.
6. Azure specific changes:
a. When reading the azure endpoint, search in a different path
and read a different option name (option-245 vs. unknown-245).
so, the lease file path should be generated according to platform.
b. adjust the handling of ephemeral mounts for ufs filesystem and
for finding the ephemeral device.
c. fix mounting of cdrom
LP: #1636345
|
|
This adds several unit tests for ds-identify, and fixes a bug
in Ec2 detection that I found while writing these tests.
The method of testing is to use the ds-identify code as a shell
library. The TestDsIdentify:call basically does:
* populate a (temp) directory with files that represent what
ds-identify would see in /sys or other locations it reads.
* create a file '_shwrap' that replaces the 3 programs that are executed
in ds-identify code path. It supports setting their stdout, stderr,
and exit code.
* set the default policies explicitly (DI_DEFAULT_POLICY) so we can
support testing different builtins. This is necessary because the
Ubuntu branches patch the builtin value. If we did not explicilty set
it, then testing there would fail.
* execute sh to source the script and call its main.
|
|
Apparently signals were crossed when this implementation was done.
Cloud-init was reading 'platform' in the environment of pid 1, but
nova-lxd was setting 'product_name'.
The fix is being made here in cloud-init to instead read product_name.
LP: #1685810
|
|
The discard action (tilde character) has been replaced by the
“stop” RainerScript directive. It is considered more intuitive and
offers slightly better performance.
The tilde operator was deprecated in rsyslog 7. Distributions
using rsyslog older than that will need to patch.
LP: #1367899
|
|
This will change all instances of LOG.warn to LOG.warning as warn
is now a deprecated method. It will also make sure any logging
uses lazy logging by passing string format arguments as function
parameters.
|
|
We were checking /var/lib/cloud/openstack/latest/meta_data.json instead
of /var/lib/cloud/seed/config_drive/openstack/latest/meta_data.json.
LP: #1673637
|
|
The reading of MAAS datasource configuration was simply broken.
it was looking in /etc/cloud/*maas*.cfg rather than
/etc/cloud/cloud.cfg.d/*maas*.cfg.
along side here there is also:
* doc improvement on check_config
* remove the path restrictions when searching for values in both
maas and ovf_vmware_guest_customization. that was done to improve
performance as check_config's parsing is slow.
* change to maas to search all config files rather than restricting
to a subset as it tried before. that was done for
* better variable names.
- rename path_cloud_confd to path_etc_cloud
- PATH_ETC_CLOUD: /etc/cloud
- PATH_ETC_CI_CFG: /etc/cloud/cloud.cfg
- PATH_ETC_CI_CFG_D: /etc/cloud/cloud.cfg.d
LP: #1677710
|
|
If the only the None datasource was listed in datasource_list, then
ds-identify would write a cloud.cfg witih:
datasource_list: [None, None]
The fix is to just append None if the list only has None.
|
|
OpenStack clouds installed with RedHat RDO have the nova product
configured in /etc/nova/release to be 'OpenStack Compute' rather than
upstream nova default of 'OpenStack Nova'.
This was first reported on Finnish provider Nebula (http://nebula.fi).
LP: #1675349
|
|
While documentation indicates that the smbios product name should
contain 'Google Compute Engine', experimentation and bug reports
indicate that is not always the case. The change here is to change
the check for GCE to also consider a serial number that starts with
'GoogleCompute-'.
Also, ds-identify was not currently searching for GCE if no config of
datasource_list was found. Most images have a datasource_list defined.
So update the list to include GCE.
LP: #1674861
|
|
This is thie cloud-init part of a fix to allow nova-lxd to provide
config drive data. The other part will be done in nova-lxd.
The agreement here is that nova-lxd will copy the contents of the
config drive to /config-drive in the container.
LP: #1673411
|
|
The path for checking presence of Bigstep datasource was simply wrong.
Set the correct path.
LP: #1674766
|
|
The script is written to have the protection of disabling filename
expansion (set -f) and explicitly enabling expansion when needed.
However, the check_config function failed to disable it after enabling.
|
|
Network configuration version 2 format is implemented in a package
called netplan (nplan)[1] which allows consolidated network config
for multiple network controllers.
- Add a new netplan renderer
- Update default policy, placing eni and sysconfig first
This requires explicit policy to enable netplan over eni
on systems which have both (Yakkety, Zesty, UC16)
- Allow any network state (parsed from any format cloud-init supports) to
render to v2 if system supports netplan.
- Move eni's _subnet_is_ipv6 to common code for use by other renderers
- Make sysconfig renderer always emit /etc/syconfig/network configuration
- Update cloud-init.service systemd unit to also wait on
systemd-networkd-wait-online.service
1. https://lists.ubuntu.com/archives/ubuntu-devel/2016-July/039464.html
|
|
Change policy so that 'report' can be overridden.
In xenial we had set the builtin default to be 'report', expecting
that Ubuntu core would install config that changed it to 'search'.
However, if report was already set, there was no way to unset it.
The change here is to make 'report' basically 'search-dryrun', so
that one or the other can be set.
The other change here is that report would actually exit disabled
if it did not find a datasource and notfound=disabled. That was
unexpected and would turn cloud-init off, which is not what we wanted.
Additionally, consistently use 'enabled' or 'disabled' versus
'enable' and 'disable'.
LP: #1669949
|
|
Rather than having the dscheck_Ec2 just know the setting, move
it up to a more formal declaration. This will make it look more
clean when a distro carries a patch to change it to warn.
|
|
On a 'not found' result, was not being written at all.
That had the unintended effect of '--force' not working.
Now, on a 'not found' result:
- if reporting: write the list as found (with just 'None').
- if not reporting: only report that there was nothing found.
this means that the warning cloud-init will write about ds-identify
failing to find a datasource will be written, but cloud-init will
still search its fully configured list.
|
|
Now, when ds-identify runs in report mode, it still writes to
/run/cloud-init.cfg as search does, but it will namespace the
result under the top level 'di_report' entry.
|
|
Z99-cloudinit-warnings.sh can be dropped into /etc/profile.d.
Warnings that are written to /var/lib/cloud/instance/warnings
will be displayed to the user on stderr when they log in.
|
|
Modify upstream packaging to install this file, which was
already installed in ubuntu packaging.
Also, white space changes from tabs to spaces. Very few things
in cloud-init are tabs now.
Lastly, remove the executable bit on this as ait is not necessary.
Scripts in /etc/profile.d do not have executable bit.
|
|
In the interest of speed I had skipped the parsing of
/etc/cloud/cloud.cfg for the ec2 strict_id setting. In hindsight
it seems reasonable for people to put settings there.
|
|
ovf_vmware_guest_customization defaults to true in cloud-init, meaning
that such customization is disabled. We just missed a return value
causing ovf_vmware_guest_customization to effectively default to on.
Also, when looking for setting look at /etc/cloud/cloud.cfg.
This had been omitted in interest of performance, but we should
be looking there.
|
|
cloud-init by default sets 'disable_vmware_customization' to True.
So in ds-identify, we will enable the ovf datasource if:
- virt is vmware
- 'libdeployPkgPlugin.so' exists as installed by vmware-tools or
open-vm-tools.
- disable_vmware_customization is configured to True
|
|
Brightbox will identify their platform to the guest by setting the
product serial to a string that ends with 'brightbox.com'.
LP: #1661693
|
|
Based on the setting Datasource/Ec2/strict_id, the datasource
will now warn once per instance.
|
|
ds-identify will now read this setting, and thus allow the user
to modify ds-identifies behavior via either:
1. builtin setting here cloud-init/ds-identify builtin
2. ds-identify config (/etc/cloud/ds-identify.cfg)
3. system config (/etc/cloud/cloud.cfg.d/*Ec2*.cfg)
4. kernel command line (ci.datasource.ec2.strict_id=true)
|
|
A check function that returns found or maybe can also now
return config that will be written to the resultant /run/cloud.cfg.
They do so by setting the variable _RET_excfg.
|
|
This just adds checking of the Ec2 seed directory.
|
|
The following can have cause issue:
FOO="bar ; wark"
showit() {
local b=$FOO
echo $b
}
4: local: ;: bad variable name
The answer is just to use more quotes.
|
|
Just remove some examples that are no longer valid.
|
|
This makes ds-identify run only once. Previously it would run multiple
times each boot as the generator would run more than once. This is
potentially dangerous, in that running again might find more attached
disks. However that is really only a "lucky" fix if it happens to
result differently than the first run.
Additionally, we now log the uptime that we started and ended at.
|
