| Age | Commit message (Collapse) | Author |
|---|
|
Move the tools/net-convert.py to be exposed as part of 'cloud-init devel'
subcommands.
It can now be called like:
$ cloud-init devel net-convert
Or, if you just have checked out source (and no cli executable):
$ python3 -m cloudinit.cmd.devel.net_convert
or
$ python3 -m cloudinit.cmd.main devel net-convert
|
|
The result of a read_file_or_url on a file and on a url would differ
in behavior.
str(UrlResponse) would return UrlResponse.contents.decode('utf-8')
while
str(FileResponse) would return str(FileResponse.contents)
The difference being "b'foo'" versus "foo".
As part of the general goal of cleaning util, move read_file_or_url
into url_helper.
|
|
With no output at all from collect-logs, users have been confused
on where the output is. By default now, write to stderr what that
file is.
Also
* add '-v' to increase verbosity. With a single -v flag, mention
what file/info is being collected.
* limit the 'journalctl' collection to this boot (--boot=0).
collecting entire journal seems unnecessary and can be huge.
* do not fail when collecting files or directories that are not there.
LP: #1766335
|
|
This enables warnings produced by pylint for unused variables (W0612),
and fixes the existing errors.
|
|
When instance meta-data provides hostname information, run
cc_set_hostname in the init-local or init-net stage before network
comes up.
Prevent an initial DHCP request which leaks the stock cloud-image default
hostname before the meta-data provided hostname was processed.
A leaked cloud-image hostname adversely affects Dynamic DNS which
would reallocate 'ubuntu' hostname in DNS to every instance brought up by
cloud-init. These instances would only update DNS to the cloud-init
configured hostname upon DHCP lease renewal.
This branch extends the get_hostname methods in datasource, cloud and
util to limit results to metadata_only to avoid extra cost of querying
the distro for hostname information if metadata does not provide that
information.
LP: #1746455
|
|
This will provide a small performance improvement and shorter code.
|
|
When we moved some tests to live under cloudinit/ we inadvertantly
failed to change all things that would run nose to include that
directory.
This changes all the 'nose' invocations to consistently run with
tests/unittests and cloudinit/.
Also, it works around, more correctly this time, a python2.6-ism with
the following code:
with assertRaises(SystemExit) as cm:
sys.exit(2)
|
|
Fix various corner cases for cloud-init status subcommand. Report
'runnning' under the following conditions:
- No /run/cloud-init/result.json file exists
- Any stage in status.json is unfinished
- status.json reports a non-null stage it is in progress on
LP: #1747965
|
|
While addressing undeclared variable in 'cloud-init status', I also fixed
the errors raised by automated code reviews against cloud-init master at
https://lgtm.com/projects/g/cloud-init/cloud-init/alerts
The following items are addressed:
* Fix 'cloud-init status':
* Only report 'running' state when any stage in
/run/cloud-init/status.json has a start time but no finished time.
Default start time to 0 if null.
* undeclared variable 'reason' now reports 'Cloud-init enabled by
systemd cloud-init-generator' when systemd enables cloud-init
* cc_rh_subscription.py util.subp return values aren't set during if an
exception is raised, use ProcessExecution as e instead.
* distros/freebsd.py:
* Drop repetitive looping over ipv4 and ipv6 nic lists.
* Initialize bsddev to 'NOTFOUND' in the event that no devs are
discovered
* declare nics_with_addresses = set() in broader scope outside
check_downable conditional
* cloudinit/util.py: Raise TypeError if mtype parameter isn't string,
iterable or None.
LP: #1744796
|
|
This issue was first identified when manual_cache_clean was set, as
ds-identify would write /run/cloud-init/cloud.cfg with
# manual_cache_clean
that would generate a warning as cloud-init expected to load a dict.
Any other "empty" config would also log such a warning.
Also fix reading of di_report to allow it to be None, as ds-identify
would write:
di_report:
# manual_cache_clean
which reads as 'di_report: None' rather than di_report: {}.
LP: #1742479
|
|
Fix cloud-init clean subcommand to unlink symlinks instead of calling
del_dir.
LP: #1741093
|
|
The cli help docs and argument parser allow the 'init' mode value
which caused a traceback.
Fix the cli to support 'init', 'config' and 'final' modes for the
cloud-init modules subcommand.
Add a check in the cli to raise a ValueError if a new
subcommand ends up allowing an unsupported/unimplemented modes.
Drive by unit test additions for a bit better coverage of error
handling.
LP: #1736600
|
|
The motivation for this is that
a.) 1.7.1 runs with python 3.6 (bionic)
b.) we want to run pylint on tests/ and tools for the same reasons
that we want to run it on cloudinit/
The changes are described below.
- Update tox.ini to invoke pylint v1.7.1.
- Modify .pylintrc generated-members ignore mocked object members (m_.*)
- Replace "dangerous" params defaulting to {}
- Fix up cloud_tests use of platforms
- Cast some instance objects to with dict()
- Handle python2.7 vs 3+ ConfigParser use of readfp (deprecated)
- Update use of assertEqual(<boolean>, value) to assert<Boolean>(value)
- replace depricated assertRegexp -> assertRegex
- Remove useless test-class calls to super class
- Assign class property accessors a result and use it
- Fix missing class member in CepkoResultTests
- Fix Cheetah test import
|
|
The 'cloud-init clean' command allows a user or script to clear cloud-init
artifacts from the system so that cloud-init sees the system as
unconfigured upon reboot. Optional parameters can be provided to remove
cloud-init logs and reboot after clean.
The 'cloud-init status' command allows the user or script to check whether
cloud-init has finished all configuration stages and whether errors
occurred. An optional --wait argument will poll on a 0.25 second interval
until cloud-init configuration is complete. The benefit here is scripts
can block on cloud-init completion before performing post-config tasks.
|
|
Add a new collect-logs sub command to the cloud-init CLI. This script
will collect all logs pertinent to a cloud-init run and store them in a
compressed tar-gzipped file. This tarfile can be attached to any
cloud-init bug filed in order to aid in bug triage and resolution.
A cloudinit.apport module is also added that allows apport interaction.
Here is an example bug filed via ubuntu-bug cloud-init: LP: #1716975.
Once the apport launcher is packaged in cloud-init, bugs can be filed
against cloud-init with the following command:
ubuntu-bug cloud-init
LP: #1607345
|
|
In an effort to save file load cost during system boot, certain
subcommands, analyze and devel, do not get loaded unless the subcommand is
specified on the commandline. Because setup.py entrypoint for cloud-init
script doesn't specify sysv_args parameter when calling the CLI's main()
we need main to read sys.argv into sysv_args so our subparser loading
continues to work.
LP: #1712676
|
|
This branch does a few things:
- Add 'schema' subcommand to cloud-init CLI for validating
cloud-config files against strict module jsonschema definitions
- Add --annotate parameter to 'cloud-init schema' to annotate
existing cloud-config file content with validation errors
- Add jsonschema definition to cc_runcmd
- Add unit test coverage for cc_runcmd
- Update CLI capabilities documentation
This branch only imports development (and analyze) subparsers when the
specific subcommand is provided on the CLI to avoid adding costly unused
file imports during cloud-init system boot.
The schema command allows a person to quickly validate a cloud-config text
file against cloud-init's known module schemas to avoid costly roundtrips
deploying instances in their cloud of choice. As of this branch, only
cc_ntp and cc_runcmd cloud-config modules define schemas. Schema
validation will ignore all undefined config keys until all modules define
a strict schema.
To perform validation of runcmd and ntp sections of a cloud-config file:
$ cat > cloud.cfg <<EOF
runcmd: bogus
EOF
$ python -m cloudinit.cmd.main schema --config-file cloud.cfg
$ python -m cloudinit.cmd.main schema --config-file cloud.cfg \
--annotate
Once jsonschema is defined for all ~55 cc modules, we will move this
schema subcommand up as a proper subcommand of the cloud-init CLI.
|
|
This branch adds cloudinit-analyze into cloud-init proper. It adds an
"analyze" subcommand to the cloud-init command line utility for quick
performance assessment of cloud-init stages and events.
On a cloud-init configured instance, running "cloud-init analyze blame"
will now report which cloud-init events cost the most wall time. This
allows for quick assessment of the most costly stages of cloud-init.
This functionality is pulled from Ryan Harper's analyze work.
The cloudinit-analyze main script itself has been refactored a bit for
inclusion as a subcommand of cloud-init CLI. There will be a followup
branch at some point which will optionally instrument detailed strace
profiling, but that approach needs a bit more discussion first.
This branch also adds:
* additional debugging topic to the sphinx-generated docs describing
cloud-init analyze, dump and show as well as cloud-init single usage.
* Updates the Makefile unittests target to include cloudinit directory
because we now have unittests within that package.
LP: #1709761
|
|
On systems with network devices with duplicate mac addresses, cloud-init
will fail to rename the devices according to the specified network
configuration. Refactor net layer to search by device driver and device
id if available. Azure systems may have duplicate mac addresses by
design.
Update Azure datasource to run at init-local time and let Azure datasource
generate a fallback networking config to handle advanced networking
configurations.
Lastly, add a 'setup' method to the datasources that is called before
userdata/vendordata is processed but after networking is up. That is
used here on Azure to interact with the 'fabric'.
|
|
Some versions of Cheetah returned everything as unicode by default (not
utf-8 or ascii) and some varieties of syslog would choke on unicode.
Jinja2 is probably fine, but Python's format() is perfectly adequate for
a short message like the welcome message.
Reviewed-by: Tom Kirchner <tjk@amazon.com>
Reviewed-by: Ben Cressey <bcressey@amazon.com>
|
|
This will change all instances of LOG.warn to LOG.warning as warn
is now a deprecated method. It will also make sure any logging
uses lazy logging by passing string format arguments as function
parameters.
|
|
This exposes a mechanism for users of cloud-init to determine if
a version has a specific feature, and adds documentation to that affect.
We list an existing feature NETWORK_CONFIG_V1 as an example.
Also add a 'features' subcommand for listing these to stdout.
|
|
If ds-identify is in report mode, and the datasource that is found
is not in the list, then warn the user of this situation.
|
|
When manual_cache_clean is enabled, write a file to
/var/lib/cloud/instance/manual-clean. That file can then be read by
ds-identify or another tool to indicate that manual cleaning is in place.
|
|
Failures to load the kernel command line's url (cloud-config-url=)
would previously get swallowed. This should make it much more
obvious when that happens. With logging going to expected places
at sane levels (WARN will go to stderr by default).
|
|
This has been a recurring ask and we had initially just made the change to
the cloud-init 2.0 codebase. As the current thinking is we'll just
continue to enhance the current codebase, its desirable to relicense to
match what we'd intended as part of the 2.0 plan here.
- put a brief description of license in LICENSE file
- put full license versions in LICENSE-GPLv3 and LICENSE-Apache2.0
- simplify the per-file header to reference LICENSE
- tox: ignore H102 (Apache License Header check)
Add license header to files that ship.
Reformat headers, make sure everything has vi: at end of file.
Non-shipping files do not need the copyright header,
but at the moment tests/ have it.
|
|
If TZ is not set in environment, then datetime.datetime() will
stat /etc/localtime on every usage. datetime is used in the logging
path of cloud-init, and as such this is very busy.
During a normal cloud-init boot, it results in we roughly
400+ stat calls to /etc/localtime.
The value we set is :/etc/localtime, which is described at
https://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html
See also merge proposal at 307722 for more background.
https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/307722
|
|
This adds a call to 'activate_datasource'. That will be called
during init stage (or init-local in the event of a 'local' dsmode).
It is present so that the datasource can do platform specific operations
that may be necessary. It is passed the fully rendered cloud-config
and whether or not the instance is a new instance.
The Azure datasource uses this to address formatting of the ephemeral
devices. It does so by
a.) waiting for the device to come online
b.) removing the marker files for the disk_setup and mounts modules
if it finds that the ephemeral device has been reset.
LP: #1611074
|
|
Change atomic_helper.write_file to have same same signature as write_file.
Add some simple unit tests for atomic_helper.
|
|
It is more efficient and cross-distribution safe to use the hooks function
from dhclient to obtain the Azure endpoint server (DHCP option 245).
This is done by providing shell scritps that are called by the hooks
infrastructure of both dhclient and NetworkManager. The hooks then
invoke 'cloud-init dhclient-hook' that maintains json data
with the dhclient options in
/run/cloud-init/dhclient.hooks/<interface>.json .
The azure helper then pulls the value from
/run/cloud-init/dhclient.hooks/<interface>.json file(s). If that file does
not exist or the value is not present, it will then fall back to the
original method of scraping the dhcp client lease file.
|
|
This moves bin/cloud-init's content into cloudinit/cmd/main.py,
and then fixes the pep8/flake8 issues with that.
The end result is easier testing of main.
|
|
|
|
|
|
|
|
This allows for the test_cli test to be more sane.
|
