| Age | Commit message (Collapse) | Author |
|---|
|
The current method is to attempt to mount the cdrom (/dev/cd0), if it is
successful, /dev/cd0 is configured, otherwise, it is not configured. The
problem is it forgets to check whether the mounting destination folder is
created or not. As a result, mounting attempt failed even if cdrom is
ready.
LP: #1696295
|
|
Some versions of Cheetah returned everything as unicode by default (not
utf-8 or ascii) and some varieties of syslog would choke on unicode.
Jinja2 is probably fine, but Python's format() is perfectly adequate for
a short message like the welcome message.
Reviewed-by: Tom Kirchner <tjk@amazon.com>
Reviewed-by: Ben Cressey <bcressey@amazon.com>
|
|
We have started adding jsonschema definitions for cloudconfig modules
(cc_ntp). This branch allows us render sphinx docs using the module's
shema definition instead of using the module's docstring.
This allows us to avoid duplicating schema documentation in the
module-level docstring and schema definition. The corresponding module
documentation is extended a bit to differentiate between config schema and
potential examples.
|
|
The comments in the debian template file of /etc/hosts still pointed
to a general template file instead of the debian one.
LP: #1606406
|
|
This just adds an entry for hostname and fqdn to 127.0.0.1 in
templates/hosts.suse.tmpl.
|
|
Unix file modes are usually represented as octal, but they were being
interpreted as decimal, for example 0o644 would be printed as '420'.
Reviewed-by: Tom Kirchner <tjk@amazon.com>
|
|
read-dependencies now takes --test-distro param to indicate we want to install
all system package depenencies to allow for testing and building for our
continous integration environment. It allows us to install all needed deps on
a fresh system with:
python3 ./tools/read-dependencies --distro ubuntu --test-distro [--dry-run].
Additionally read-dependencies now looks at what version of python is running
the script (py2 vs p3) and opts to install python 2 or 3 system deps
respectively. This behavior can still be overridden with
python3 ./tools/read-dependencies ... --python-version 2.
There are also some distro-specific packaging and test dependencies, like
devscripts, tox and libssl-dev on debian or ubuntu. Those pkg dependencies
have now been broken out from common pkg deps to avoid trying to install them
on centos/redhat/suse.
|
|
These changes are all in an effort to get tools/run-centos using
read-dependencies rather than the 'setup-centos' script with a separate
set of dependencies listed.
- tools/read-dependencies: support taking multiple --requirements
options. This allows run-centos to get both test and build
dependencies. Ultimately, I think it might be nicer for
read-dependencies to take a list of "goals" (build, test, run or
test-tox) rather than having the caller need to know to provide
multiple --requirements.
- packages/pkg-deps.json: drop the version on the sudo package.
centos 6 has newer (1.8.6p3) version than listed, so its not a problem.
- test_handler_disk_setup.py: a test case here was using assertLogs
which is not present in the version of unittest2 that is available in
centos 6 epel. We just adjust it to use with_logs = True.
- tools/run-cents:
- improve usage with example
- add 'inside_as_cd' to provide the dir you want to cd first to.
- avoid the intermediate tarball on disk in the container.
- add 'prep' subcommand and use it to install pre-dependencies.
- use read-dependencies.
|
|
This change adds a couple of makefile targets for ci environments to
install all necessary dependencies for package builds and test runs.
It adds a number of arguments to ./tools/read-dependencies to facilitate
reading pip dependencies, translating pip deps to system package names and
optionally installing needed system-package dependencies on the local
system. This relocates all package dependency and translation logic into
./tools/read-dependencies instead of duplication found in packages/brpm
and packages/bddeb.
In this branch, we also define buildrequires as including all runtime
requires when rendering cloud-init.spec.in and debian/control files
because our package build infrastructure will also be running all unit
test during the package build process so we need runtime deps at build
time.
Additionally, this branch converts
packages/(redhat|suse)/cloud-init.spec.in from cheetah templates to jinja
to allow building python3 envs.
|
|
This changes all cloud-init systemd units to run 'Before' the apt processes
that run daily and may cause a lock on the apt database.
apt-daily-upgrade.service contains 'After=apt-daily.service'.
Thus following order is enforced, so we can just be 'Before' the first.
apt-daily.service
apt-daily-upgrade.service
Note that this means only that apt-daily* will not run until
cloud-init has entirely finished. Any other processes running apt-get
operations are still affected by the global lock.
LP: #1693361
|
|
On some systems with python-libselinux a bug[1] related to recursive
restorecon fails but the distro release does not yet include
an update. This change will accept the error and log a warning.
1. https://bugzilla.redhat.com/show_bug.cgi?id=1406520
LP: #1686751
|
|
On systems with selinux enabled, some of the networking commands executed
successfully do not return 0. Allow these commands to return 1 since the
output is valid.
Ultimately we need to get this information in some way so that we can
display it correctly. For now, work around the stack trace when selinux
does not allow us to collect it.
LP: #1686751
|
|
In cases where the config json specifies nameserver entries,
if there are interfaces configured to use dhcp, NetworkManager,
if enabled, will clobber the /etc/resolv.conf that cloud-init
has produced, which can break dns. If there are no interfaces
configured to use dhcp, NetworkManager could clobber
/etc/resolv.conf with an empty file.
This patch adds a mechanism for dropping additional configuration
into /etc/NetworkManager/conf.d/ and disables management of
/etc/resolv.conf by NetworkManager when nameserver information is
provided in the config.
LP: #1693251
Signed-off-by: Ryan McCabe <rmccabe@redhat.com>
|
|
The typical rpm build process will examine the spec file to determine
which packages should be installed in the boot root. This requires
the specfile to declare that it needs system. Provide this information
by checking which version in which the rpm is being built and exporting
requirements for systemd.
|
|
The network_state object's network and route keys would have different
information depending upon how the network_state object was populated.
This change cleans that up. Now:
* address will always contain an IP address.
* prefix will always include an integer value that is the
network_prefix for the address.
* netmask will be present only if the address is ipv4, and its
value will always correlate to the 'prefix'.
|
|
Massive update to clean up and greatly enhance the integration testing
framework developed by Wesley Wiedenmeier.
- Updated tox environment to run integration test 'citest' to utilize
pylxd 2.2.3
- Add support for distro feature flags
- add framework for feature flags to release config with feature groups
and overrides allowed in any release conf override level
- add support for feature flags in platform and config handling
- during collect, skip testcases that require features not supported by
the image with a warning message
- Enable additional distros (i.e. centos, debian)
- Add 'bddeb' command to build a deb from the current working tree
cleanly in a container, so deps do not have to be installed on host
- Adds a command line option '--preserve-data' that ensures that
collected data will be left after tests run. This also allows the
directory to store collected data in during the run command to be
specified using '--data-dir'.
- Updated Read the Docs testing page and doc strings for pep 257
compliance
|
|
- Updated to standard chef.io url
- Removed the port 4000, due to that has been deprecated
- Added Note about the run_list not being required
Signed-off-by: JJ Asghar <jj@chef.io>
|
|
The added 'run-centos' does:
- Creates centos 6 or 7 lxd container
* Sets http_proxy variable for yum if set locally
* Creates centos user
- Push local tree
* Tar's up working directory
* Pushes to container and untars
- Installs pip and yum dependencies
- As user centos it can then based on flags:
* runs unittests
* run ./packages/brpm
* run ./packages/brpm --srpm
* artifact the built *.rpm
|
|
Cloud-init's template renderer does not require the use of cheetah
(which is python2 only) so do not put it in the list of package
requirements.
|
|
Many changes here to get us able to build rpms on CentOS 5 or 6 and RHEL.
* add 'Requires' as 'BuildRequires' also.
This allows us to run cloud-init tools in the build environment, and
also will allow us to run tests in the build process.
* build for both systemd and upstart (centos 5) init systems.
* Add 'centos' as a variant
Adding the variant means we can use the 'centos' user as default on centos
rather than a 'fedora' or 'rhel'.
* drop argparse from the requirements.
On any system other than python 2.6, having a 'requirements' that mentions
argparse just causes problems. Instead we add that Requires to the spec
directly.
* list dependency on dmidecode (as redhat distro spec had)
* remove duplicate line in files section ({_unitdir}/cloud-*)
* Use rpm macros for init-system chunks and drop use
of init_system variable template
* Add el6 only build-req on python-argparse
* python-cheetah is not required in the build environment as the
the spec is already rendered. (We will soon move the spec to jinja).
|
|
Here we move the config/cloud.cfg to be rendered as a template.
That allows us to maintain deltas between distros in one place.
Currently we use 'variant' variable to make decisions.
A tools/render-cloudcfg is provided to render the file.
There were changes to setup.py, MANIFEST.in to allow us to put all
files into a virtual env installation and to render the cloud-config
file in 'install' or 'bdist' targets.
We have also included some config changes that were found in the
redhat distro spec.
* include some config changes from the redhat distro spec.
The rendered cloud.cfg has some differences.
Ubuntu: white space and comment changes only.
Freebsd:
- whitespace changes and comment changes
- datasource_list definition moved to be closer to 'datasource'.
- enable modules: migrator, write_files
- move package-update-upgrade-install to final.
The initial work was done by Josh Harlow.
|
|
This just adds targets for deb-src and srpm, and uses
PYVER anywhere where we run a python program.
|
|
Fix detection of python in a non-python3 environment.
The old path always used python3. The 2 fixes here are:
a.) escape the '$' before the subshell.
b.) use shell builtin 'command -v' rather than 'which' in case
'which' is not available.
|
|
The snap is a classic snap and does not need to specify any plugs
because it is already unconfined.
LP: #1695333
|
|
Since f38fa413176, default routes get added to both ifcfg-* and route-*
and route6-* files. Default routes should only go to ifcfg-* files,
otherwise the information is redundant.
LP: #1696176
|
|
The jsonschema package is used only when available, but the lint
check thinks the import is unused across pyflakes and flake8. In
order to avoid having exceptions for both assert that the import
works right after and the import is considered used.
The '# NOQA' doesn't affect pyflakes (only flake8).
LP: #1695918
|
|
This allows the user to seed NoCloud in a trivial way from qemu/libvirt,
by using a stock image and passing a single command line flag. No custom
command line, no filesystem modification, no bootstrap disk image.
This is particularly handy now that Ec2 backend is discouraged from use
under bug 1660385.
LP: #1691772
|
|
The code deciding which interface to choose as the default to request the
IP address through DHCP does not sort the interfaces correctly. On Ubuntu
Xenial images for example, the interfaces are named ens1, ens2, ens3...,
ens11, ... depending on the pci bus address. The python sorting will list
'ens11' before 'ens3' for example despite the fact that 'ens3' should be
before 'ens11'.
This patch address this issue and sort the interface names according to a
human sorting.
Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>
|
|
|
|
Simply fix a typo in a comment. No functional change.
|
|
On some build environments we don't have python-jsonschema installed.
Since this dependency is an optional runtime dependency, we can also make
it an optional unit test dependency. Add a skip of related unittests when
jsonschema is not present.
Also, KeyError messages on CentOs don't have single quotes around the
missing 'key-name'. Make our KeyError assertion a bit more flexible with
the assertIn call.
LP: #1695318
|
|
In previous commit I inadvertantly left two calls to
asset_tag = util.read_dmi_data('chassis-asset-tag')
The second did not do anything useful. Thus, remove it.
|
|
Azure sets a known chassis asset tag to 7783-7084-3265-9085-8269-3286-77.
We can inspect this in both ds-identify and DataSource.get_data to
determine whether we are on Azure.
Added unit tests to cover these changes
and some minor tweaks to Exception error message content to give more
context on malformed or missing ovf-env.xml files.
LP: #1693939
|
|
Older cloud-init versions have a bug in the signature of the
render_network_state method for netplan (bug 1685944).
The old had:
render_network_state(target, network_state)
The fix was to change netplan's so it had the correct signature:
render_network_state(network_state, target)
This just changes our caller to use kwargs style when invoking that
method so that it works with either the broken form or correct form.
|
|
cloud-config files are very flexible and permissive.
This adds a jsonsschema definition to the cc_ntp module and validation
functions in cloudinit/config/schema which will log warnings about
invalid configuration values in the ntp section.
A cmdline tools/cloudconfig-schema is added which can be used in our dev
environments to quickly attempt to exercise the ntp schema.
It is also exposed as a main in cloudinit.config.schema.
(python3 -m cloudinit.config.schema)
LP: #1692916
|
|
There are a few bridge parameters which require repeating the key with each
value in the list when rendering eni. Extend the network unittests to cover
all of the known bridge parameters and check we render eni and netplan
correctly.
|
|
During the network v2 merge, we inadvertently re-enabled rendering systemd
.link files. This files are not required as cloud-init already has to do
interface renaming due to issues with udevd which may refuse to rename
certain interfaces (such as veth devices in a LXD container). As such,
removing the code altogether.
|
|
AliYun cloud platform is now identifying themselves by setting the dmi
product id to the well known value "Alibaba Cloud ECS". The changes here
identify that properly in tools/ds-identify and in the DataSourceAliYun.
Since the 'get_data' for AliYun now identifies itself correctly, we can
enable AliYun by default.
LP: #1638931
|
|
Input (specifically OpenStack) that had:
"ip_address" : "104.130.20.155",
"netmask" : "255.255.255.0"
Was being rendered to netplan as '104.130.20.155/255.255.255.0'.
That is now fixed to '104.130.20.155/24'
Also fixed is reading of a route that had a network prefix integer
in the 'netmask' rather than a netmask.
LP: #1689346
LP: #1684349
|
|
This attempts to use udevadm settle to wait until devices have been
fully "realized". If a device exists, there may still be events in
the udev queue that would create its partition table entries.
We need to wait until those have been processed also.
LP: #1692093
|
|
Per the documentation at
https://cloud.google.com/compute/docs/storing-retrieving-metadata
The instance-level SSH key was named 'sshKeys' and now is 'ssh-keys'.
The project-level SSH key attribute has not changed so is intentionally
not changed here.
LP: #1693582
|
|
As some of the bond paramemters are passed in as dashed, or
underscored, depending on the input source.
Also correct transmit-hash-policy netplan target key.
LP: #1690480
|
|
python2.6 unittest.TestCase does not have the assertIsNone or
assertIsNotNone. We just have to explicitly use the unittest2
version, which we get from helpers.
The desire to use assertIsNone comes from flake8 (through hacking,
I believe).
Also, fix "{}.format('foo')" which is not valid in python2.6.
|
|
Simply improve the docstring on a test added in last commit.
|
|
If the user configured:
datasource_list: ["Ec2", "None"]
then ds-identify would write
datasource_list: ["Ec2", "None", "None"]
which would break the logic to avoid warning.
|
|
When the style/checking dependencies were updated in
test-requirements.txt, the debian package build dependencies created
by ./packages/bddeb were also updated. Pycodestyle was added to the list
in order to pin its version. That broke the package build for 16.04. The
reason for this is simply that python3-pycodestyle is not available in
16.04.
The change here is to remove style dependencies from test-requirements,
and add them to the tox environments directly. We had previously changed
the package build process to not run pep8 or flake8 simply to avoid having
to code to N different versions of style checkers (3bcb72c593f).
The link between package build and test-requirements still exists, though.
So future breakage can occur if any package is added to
test-requirements.txt (or requirements.txt) if the target distro release
does not have a python3-<packagename> in its archive.
There is also a bit of a tox.ini cleanup here, in that we do not have to
explictly list '-rrequirements.txt' as the setup.py pulls those in. And
lastly, we drop the -rtest-requirements.txt from the base 'testenv', and
add these test requirements only to environments that need to run test.
Finally, a change to packages/debian/control.in to drop the build
dependencies that were listed for style checking and also a dependency
on iproute2 which was a bad unit test that has been previously fixed.
|
|
Any CiTestCase subclass can now set a class attribute with_logs = True and
tests can now make assertions on self.logs.getvalue(). This branch
restructures a bit of cc_ntp module to get better test coverage of the
module. It also restructures the handler_cc_ntp unit tests to avoid nested
mocks where possible. Deeply nested mocks cause a couple of issues:
- greater risk: mocks are permanent within the scope, so multiple
call-sites could be affected by package mocks
- less legible tests: each mock doesn't advertise the actual call-site
- tight coupling: the unit test logic to tightly bound to the actual
implementation in remote (unrelated) modules which makes it more
costly to maintain code
- false success: we should be testing the expected behavior not specific
remote method names as we want to know if that underlying behavior
changes and breaks us.
LP: #1692794
|
|
This just moves flake8 and related tools up to newer versions and fixes
the complaints associated with that.
We added to the list of flake8 ignores:
H102: do not put vim info in source files
H304: no relative imports
Also updates and pins the following in the flake8 environment:
pep8: 1.7.0 => drop (although hacking still pulls it in).
pyflakes 1.1.0 => 1.5.0
hacking 0.10.2 => 0.13.0
flake8 2.5.4 => 3.3.0
pycodestyle none => 2.3.1
|
|
Snapd does not start on artful or on the versions in proposed. This
changes the behavior of the test to confirm that snapd is installed, not
that it is started, while the snap team fixes the issue (LP: ##1690880).
|
|
Dual stack IPv4/IPv6 configuration via config drive is broken for RHEL7.
This patch fixes several scenarios for IPv4/IPv6/dual-stack with multiple
IP assignment.
Removes usage of unpopular IPv4 alias files and invalid IPv6 alias files.
Also fix associated unit tests.
LP: #1679817
LP: #1685534
LP: #1685532
|
