| Age | Commit message (Collapse) | Author |
|---|
|
This was painful, but it finishes a TODO from cloudinit/subp.py.
It moves the following from util to subp:
ProcessExecutionError
subp
which
target_path
I moved subp_blob_in_tempfile into cc_chef, which is its only caller.
That saved us from having to deal with it using write_file
and temp_utils from subp (which does not import any cloudinit things now).
It is arguable that 'target_path' could be moved to a 'path_utils' or
something, but in order to use it from subp and also from utils,
we had to get it out of utils.
|
|
Since upstream cloud-init has dropped python2 support,
adapt remaining package build scripts and tools to python3 only
Changes:
* Do not template debian/rules as python3 is the only supported version
* Drop six from requirements.txt
* Makefile: drop everything related to Python 2
* run-container: install the CI deps only on ubuntu|debian
* read-version: update the shebang to use Python 3
* brpm: read_dependencies(): drop unused argument
* read-dependencies: switch to Py3 and drop the --python-version option
* pkg-deps.json: drop the Python version field and update the redhat deps
* pkg-deps.json: drop the unittest2 and contextlib2 renames
* Update RPM the spec file to use Python 3 when building the RPM
* bddeb: drop support for Python 2
|
|
These libraries provide backports of Python 3's stdlib components to Python 2. As we only support Python 3, we can simply use the stdlib now. This pull request does the following:
* removes some unneeded compatibility code for the old spelling of `assertRaisesRegex`
* replaces invocations of the Python 2-only `assertItemsEqual` with its new name, `assertCountEqual`
* replaces all usage of `unittest2` with `unittest`
* replaces all usage of `contextlib2` with `contextlib`
* drops `unittest2` and `contextlib2` from requirements files and tox.ini
It also rewrites some `test_azure` helpers to use bare asserts. We were seeing a strange error in xenial builds of this branch which appear to be stemming from the AssertionError that pytest produces being _different_ from the standard AssertionError. This means that the modified helpers weren't behaving correctly, because they weren't catching AssertionErrors as one would expect. (I believe this is related, in some way, to https://github.com/pytest-dev/pytest/issues/645, but the only version of pytest where we're affected is so far in the past that it's not worth pursuing it any further as we have a workaround.)
|
|
|
|
As the nose docs[0] themselves note, it has been in maintenance mode for the past several years. pytest is an actively developed, featureful and popular alternative that the nose docs themselves recommend. See [1] for more details about the thinking here.
(This PR also removes stale tox definitions, instead of modifying them.)
[0] https://nose.readthedocs.io/en/latest/
[1] https://lists.launchpad.net/cloud-init/msg00245.html
|
|
* tools/read-version: don't enforce version parity in release branch CI
We have a bootstrapping problem with new releases, currently. To take
the example of 20.1: the branch that bumps the version fails CI because
there is no 20.1 tag for it to use in read-version. Previously, this
was solved by creating a tag and pushing it to the cloud-init repo
before the commit landed. However, we have GitHub branch protection
enabled, so the commit that needs to be tagged is not created until the
pull request lands in master.
This works around this problem by introducing a very specific check: if
we are performing CI for an upstream release branch, we skip the
read-version checking that we know will fail.
* tools/make-tarball: add --version parameter
When using make-tarball as part of a CI build of a new upstream release,
the version it determines is inconsistent with the version that other
tools determine. Instead of encoding the logic here (as well as in
Python elsewhere), we add a parameter to allow us to set it from outside
the script.
* packages/bddeb: handle missing version_long in new version CI
If we're running in CI for a new upstream release, we have to use
`version` instead of `version_long` (because we don't yet have the tag
required to generate `version_long`).
|
|
* packages/brpm: switch to python3
No changes needed other than changing the shebang interpreter.
* pkg-deps.json: bump the redhat build dependencies to python36
CentOS 7 (our standard target for the COPR test builds) uses python3.6
as its default python3 interpreter, so let's bump the build dependencies
accordingly.
|
|
|
|
LP: #1825444
|
|
|
|
|
|
When invoked with '--distro=suse', the packages that would be
attempted for installation would be from redhat. We just were not
pasing the args.distro through. That is fixed here.
|
|
This makes the necessary changes to patch the full packaged version into
the trunk maintained redhat and suse spec files.
|
|
This modifies version.version_string to support having the package
build write the *packaged* version in with a easy replace.
Then, when cloud-init reports its version it will include the full
packaged version.
Also modified here are upstream package build files to get that done.
Note part of the trickery in packages/debian/rules.in was to avoid
the 'basic' templater consuming the '$variable' variable names.
LP: #1770712
|
|
bddeb already supported passing in a '--release' and that would get
into the changelog line.
If you used bddeb to build packages for a PPA, and built multiple
releases, then you would get the same version for each release, and
launchpad would reject your upload.
The change here means we get a ~16.04.1 (for xenial) suffix on the
dpkg version. If the distro-info-data package is not installed,
or the release is not known (such as the default "UNRELEASED"),
then you get no suffix.
|
|
If you built packages with 'bddeb', each time it would create a new
tarball with make-tarball. If you then tried to upload two different
tarballs to launchpad (to a PPA), it would reject the second as the
orig tarball already existed.
This just supports looking in some places for a orig tarball and
re-using if it is found.
|
|
Ubuntu minimal images do not have iproute2, so correctly identify
our dependency on it.
LP: #1766711
|
|
In bash shells with bash_completion enabled, now the cloud-init
sub commands and parameters/flags will be shown.
|
|
This just correctly adds the missing dependency on isc-dhcp-client.
That package is used via 'dhclient' from cloudinit/net/dhcp.py.
LP: #1759307
|
|
boto_utils.py had been removed some time ago, and the current
cloudinit/ec2_utils.py is not based on what was in boto_utils.
We just failed to remove the mention of it from the upstream
debian/copyright. And then put it back in everywhere in recent changes
to get upstream and ubuntu in sync.
|
|
The first revision of this rendered tables with less decoration but there
was a desire upstream to avoid possibly breaking some parsing someone
might be doing, so it has been revised to render the same as prettytable
for the cases cloud-init actually uses.
|
|
The copyright was updated to be lintian clean and reorganized to list the
licenses at the bottom after declaring the metadata and file information.
Add the MIT license to the file.
LP: #1718681
|
|
Add a new collect-logs sub command to the cloud-init CLI. This script
will collect all logs pertinent to a cloud-init run and store them in a
compressed tar-gzipped file. This tarfile can be attached to any
cloud-init bug filed in order to aid in bug triage and resolution.
A cloudinit.apport module is also added that allows apport interaction.
Here is an example bug filed via ubuntu-bug cloud-init: LP: #1716975.
Once the apport launcher is packaged in cloud-init, bugs can be filed
against cloud-init with the following command:
ubuntu-bug cloud-init
LP: #1607345
|
|
Revert "centos: do not package systemd-fsck drop-in."
Revert "systemd: make systemd-fsck run after cloud-init.service"
The systemd-fsck drop-in caused regressions by introducing ordering
The change reverts the original commit that added systemd-fsck drop-in
and another commit that had removed that from the centos packaging:
1f5489c258a26f4e26261c40786537951d67df1e
8a5296c41db45be3a172862f324ad44e732a2250
The result is to no longer provide the systemd-fsck drop-in.
LP: #1717477
|
|
During boot, the usage of /tmp is not safe. In systemd systems,
systemd-tmpfiles-clean may run at any point and clear out a temp file
while cloud-init is using it. The solution here is to use
/run/cloud-init/tmp.
LP: #1707222
|
|
The ubuntu-init-switch module allowed the use to launch an instance that
was booted with upstart and have it switch its init system to systemd and
then reboot itself. It was only useful for the time period when Ubuntu was
transitioning to systemd but only produced images using upstart.
Also, do not run setup with --init-system=upstart. This means that by
default, debian packages built with packages/bddeb will not have upstart
unit files included. No other removal is done here.
|
|
The change here fixes the build of CentOS 7 rpm.
The systemd 'drop-in' file systemd-fsck@.service.d/cloud-init.conf is
not expected to work on CentOS at the moment. There, the
cloud-init-local.service currently runs without DefaultDependencies=no.
That would likely cause a dependency loop.
The drop-in was added for bug 1691489. It may well be needed at a later
date in CentOS also.
|
|
Under el7, cloud-init systemd files need some unit tweaks to ensure
they run at the right time. Pull in current el7 downstream systemd unit
changes.
|
|
read-dependencies now takes --test-distro param to indicate we want to install
all system package depenencies to allow for testing and building for our
continous integration environment. It allows us to install all needed deps on
a fresh system with:
python3 ./tools/read-dependencies --distro ubuntu --test-distro [--dry-run].
Additionally read-dependencies now looks at what version of python is running
the script (py2 vs p3) and opts to install python 2 or 3 system deps
respectively. This behavior can still be overridden with
python3 ./tools/read-dependencies ... --python-version 2.
There are also some distro-specific packaging and test dependencies, like
devscripts, tox and libssl-dev on debian or ubuntu. Those pkg dependencies
have now been broken out from common pkg deps to avoid trying to install them
on centos/redhat/suse.
|
|
These changes are all in an effort to get tools/run-centos using
read-dependencies rather than the 'setup-centos' script with a separate
set of dependencies listed.
- tools/read-dependencies: support taking multiple --requirements
options. This allows run-centos to get both test and build
dependencies. Ultimately, I think it might be nicer for
read-dependencies to take a list of "goals" (build, test, run or
test-tox) rather than having the caller need to know to provide
multiple --requirements.
- packages/pkg-deps.json: drop the version on the sudo package.
centos 6 has newer (1.8.6p3) version than listed, so its not a problem.
- test_handler_disk_setup.py: a test case here was using assertLogs
which is not present in the version of unittest2 that is available in
centos 6 epel. We just adjust it to use with_logs = True.
- tools/run-cents:
- improve usage with example
- add 'inside_as_cd' to provide the dir you want to cd first to.
- avoid the intermediate tarball on disk in the container.
- add 'prep' subcommand and use it to install pre-dependencies.
- use read-dependencies.
|
|
This change adds a couple of makefile targets for ci environments to
install all necessary dependencies for package builds and test runs.
It adds a number of arguments to ./tools/read-dependencies to facilitate
reading pip dependencies, translating pip deps to system package names and
optionally installing needed system-package dependencies on the local
system. This relocates all package dependency and translation logic into
./tools/read-dependencies instead of duplication found in packages/brpm
and packages/bddeb.
In this branch, we also define buildrequires as including all runtime
requires when rendering cloud-init.spec.in and debian/control files
because our package build infrastructure will also be running all unit
test during the package build process so we need runtime deps at build
time.
Additionally, this branch converts
packages/(redhat|suse)/cloud-init.spec.in from cheetah templates to jinja
to allow building python3 envs.
|
|
The typical rpm build process will examine the spec file to determine
which packages should be installed in the boot root. This requires
the specfile to declare that it needs system. Provide this information
by checking which version in which the rpm is being built and exporting
requirements for systemd.
|
|
Many changes here to get us able to build rpms on CentOS 5 or 6 and RHEL.
* add 'Requires' as 'BuildRequires' also.
This allows us to run cloud-init tools in the build environment, and
also will allow us to run tests in the build process.
* build for both systemd and upstart (centos 5) init systems.
* Add 'centos' as a variant
Adding the variant means we can use the 'centos' user as default on centos
rather than a 'fedora' or 'rhel'.
* drop argparse from the requirements.
On any system other than python 2.6, having a 'requirements' that mentions
argparse just causes problems. Instead we add that Requires to the spec
directly.
* list dependency on dmidecode (as redhat distro spec had)
* remove duplicate line in files section ({_unitdir}/cloud-*)
* Use rpm macros for init-system chunks and drop use
of init_system variable template
* Add el6 only build-req on python-argparse
* python-cheetah is not required in the build environment as the
the spec is already rendered. (We will soon move the spec to jinja).
|
|
When the style/checking dependencies were updated in
test-requirements.txt, the debian package build dependencies created
by ./packages/bddeb were also updated. Pycodestyle was added to the list
in order to pin its version. That broke the package build for 16.04. The
reason for this is simply that python3-pycodestyle is not available in
16.04.
The change here is to remove style dependencies from test-requirements,
and add them to the tox environments directly. We had previously changed
the package build process to not run pep8 or flake8 simply to avoid having
to code to N different versions of style checkers (3bcb72c593f).
The link between package build and test-requirements still exists, though.
So future breakage can occur if any package is added to
test-requirements.txt (or requirements.txt) if the target distro release
does not have a python3-<packagename> in its archive.
There is also a bit of a tox.ini cleanup here, in that we do not have to
explictly list '-rrequirements.txt' as the setup.py pulls those in. And
lastly, we drop the -rtest-requirements.txt from the base 'testenv', and
add these test requirements only to environments that need to run test.
Finally, a change to packages/debian/control.in to drop the build
dependencies that were listed for style checking and also a dependency
on iproute2 which was a bad unit test that has been previously fixed.
|
|
Add a simple dependency check to "make deb" target for devscripts. Rework
a bit of the logic in package/bddeb to drop superfluous STD_NAMED_PACKAGES
to avoid duplication of requirements already listed in
(test-)?requiremets.txt. All "standard" packages can be assumed to have
either python3- or python- prefix if not listed in NONSTD_NAMED_PACKAGES.
This branch also moves logic inside write_debian_folder which is unneeded
up in main.
LP: #1685935
|
|
Z99-cloudinit-warnings.sh can be dropped into /etc/profile.d.
Warnings that are written to /var/lib/cloud/instance/warnings
will be displayed to the user on stderr when they log in.
|
|
Modify upstream packaging to install this file, which was
already installed in ubuntu packaging.
Also, white space changes from tabs to spaces. Very few things
in cloud-init are tabs now.
Lastly, remove the executable bit on this as ait is not necessary.
Scripts in /etc/profile.d do not have executable bit.
|
|
This has been a recurring ask and we had initially just made the change to
the cloud-init 2.0 codebase. As the current thinking is we'll just
continue to enhance the current codebase, its desirable to relicense to
match what we'd intended as part of the 2.0 plan here.
- put a brief description of license in LICENSE file
- put full license versions in LICENSE-GPLv3 and LICENSE-Apache2.0
- simplify the per-file header to reference LICENSE
- tox: ignore H102 (Apache License Header check)
Add license header to files that ship.
Reformat headers, make sure everything has vi: at end of file.
Non-shipping files do not need the copyright header,
but at the moment tests/ have it.
|
|
Adjust the redhat spec file to fix errors found during a ./tools/brpm
on centos 6:
RPM build errors:
File listed twice: /usr/libexec/cloud-init/uncloud-init
File listed twice: /usr/libexec/cloud-init/write-ssh-key-fingerprints
Installed (but unpackaged) file(s) found:
/etc/NetworkManager/dispatcher.d/hook-network-manager
/etc/dhcp/dhclient-exit-hooks.d/hook-dhclient
|
|
When we added coverage to test-requirements, we need to add the mapping
to package name. Without it there, bddeb complains that it cannot
translate the dependency.
Note, though, that the Makefile does not invoke nose with coverage.
So we don't actually use that dependency.
|
|
./packages/bddeb --release=xenial
that will get you a changelog with Distribution of xenial
rather than UNRELEASED.
|
|
For upstream version directly use the output of git-describe
(X.Y.Z-number.gHASH) rather than rather than changing it to
(X.Y.Z+number.gHASH).
The rpm version does not allow '-' in Version, so we create and use
rpm_upstream_version in the rpm spec file. That is of format:
X.Y.Z+number.gHASH
|
|
Modification of the tarball became problematic, as it meant that
any tool extracting source would find the orig source tarball different.
I found this unusable when trying to use 'gbp buildpackage'.
Other changes here are to better support using python3 or python2
for the build. Makefile will try to call the right python version
and can be told which python to use.
read-version: by adding 'tiny_p' and avoiding the import of
cloudinit.util, we need less dependencies to run this.
|
|
upstream snapshots are versioned in the format 'X.Y.Z+<distance>.g<commit>'
where X.Y.Z are major, minor, and micro. Distance is number of commits
since last annotated tag, and commit is the git commit.
bddeb and brpm will now create and use the "upstream version" like above.
Things changed here:
- tools/make-tarball
update cloudinit/version.py to contain the full version
support --output
support '--long' to always create the long format version string.
- bddeb:
- use quilt debian source format
- use read-version and long version in changelog.
- brpm:
- change to use read-version and upstream long version in the spec.
- flake8 changes
- tools/read-version
- read version from git or from cloudinit/version.
- provide --json output with more nicely formed data.
|
|
- Update HACKING.rst to include git instructions
- update MANIFEST.in and .gitignore to ignore git-related things
- replaced tarball generation scripts with git-based script
- have the spec files correctly identify themselves as cheetah templates
- make brpm work with git
|
|
Its not needed since its a built-in for these
versions so we can just skip it for any newer
rpm building in the first place.
LP: #1603533
|
|
These new files were not getting picked up during
packaging (and they need to, otherwise rpm building
fails).
- 66-azure-ephemeral.rules
- cloud-init-generator
|
|
Splits off distro specific code into specific files so that
other kinds of networking configuration can be written by the
various distro(s) that cloud-init supports.
It also isolates some of the cloudinit.net code so that it can
be more easily used on its own (and incorporated into other
projects such as curtin).
During this process it adds tests so that the net process can
be tested (to some level) so that the format conversion processes
can be tested going forward.
|
|
|
|
|
