summaryrefslogtreecommitdiff
path: root/tools
AgeCommit message (Collapse)Author
2017-04-21pylint: fix all logging warningsJoshua Powers
This will change all instances of LOG.warn to LOG.warning as warn is now a deprecated method. It will also make sure any logging uses lazy logging by passing string format arguments as function parameters.
2017-04-14ds-identify: Check correct path for "latest" config driveDaniel Watkins
We were checking /var/lib/cloud/openstack/latest/meta_data.json instead of /var/lib/cloud/seed/config_drive/openstack/latest/meta_data.json. LP: #1673637
2017-03-30ds-identify: fix detecting of maas datasource.Scott Moser
The reading of MAAS datasource configuration was simply broken. it was looking in /etc/cloud/*maas*.cfg rather than /etc/cloud/cloud.cfg.d/*maas*.cfg. along side here there is also: * doc improvement on check_config * remove the path restrictions when searching for values in both maas and ovf_vmware_guest_customization. that was done to improve performance as check_config's parsing is slow. * change to maas to search all config files rather than restricting to a subset as it tried before. that was done for * better variable names. - rename path_cloud_confd to path_etc_cloud - PATH_ETC_CLOUD: /etc/cloud - PATH_ETC_CI_CFG: /etc/cloud/cloud.cfg - PATH_ETC_CI_CFG_D: /etc/cloud/cloud.cfg.d LP: #1677710
2017-03-29ds-identify: do not write None twice to datasource_list.Scott Moser
If the only the None datasource was listed in datasource_list, then ds-identify would write a cloud.cfg witih: datasource_list: [None, None] The fix is to just append None if the list only has None.
2017-03-28OpenStack: identify OpenStack by product 'OpenStack Compute'.Scott Moser
OpenStack clouds installed with RedHat RDO have the nova product configured in /etc/nova/release to be 'OpenStack Compute' rather than upstream nova default of 'OpenStack Nova'. This was first reported on Finnish provider Nebula (http://nebula.fi). LP: #1675349
2017-03-27GCE: Search GCE in ds-identify, consider serial number in check.Scott Moser
While documentation indicates that the smbios product name should contain 'Google Compute Engine', experimentation and bug reports indicate that is not always the case. The change here is to change the check for GCE to also consider a serial number that starts with 'GoogleCompute-'. Also, ds-identify was not currently searching for GCE if no config of datasource_list was found. Most images have a datasource_list defined. So update the list to include GCE. LP: #1674861
2017-03-24ConfigDrive: support reading config drive data from /config-drive.Scott Moser
This is thie cloud-init part of a fix to allow nova-lxd to provide config drive data. The other part will be done in nova-lxd. The agreement here is that nova-lxd will copy the contents of the config drive to /config-drive in the container. LP: #1673411
2017-03-24ds-identify: fix detection of Bigstep datasource.Scott Moser
The path for checking presence of Bigstep datasource was simply wrong. Set the correct path. LP: #1674766
2017-03-22ds-identify: fix bug where filename expansion was left on.Scott Moser
The script is written to have the protection of disabling filename expansion (set -f) and explicitly enabling expansion when needed. However, the check_config function failed to disable it after enabling.
2017-03-20cloudinit.net: add network config v2 parsing and renderingRyan Harper
Network configuration version 2 format is implemented in a package called netplan (nplan)[1] which allows consolidated network config for multiple network controllers. - Add a new netplan renderer - Update default policy, placing eni and sysconfig first This requires explicit policy to enable netplan over eni on systems which have both (Yakkety, Zesty, UC16) - Allow any network state (parsed from any format cloud-init supports) to render to v2 if system supports netplan. - Move eni's _subnet_is_ipv6 to common code for use by other renderers - Make sysconfig renderer always emit /etc/syconfig/network configuration - Update cloud-init.service systemd unit to also wait on systemd-networkd-wait-online.service 1. https://lists.ubuntu.com/archives/ubuntu-devel/2016-July/039464.html
2017-03-06ds-identify: report cleanups for config and exit value.Scott Moser
Change policy so that 'report' can be overridden. In xenial we had set the builtin default to be 'report', expecting that Ubuntu core would install config that changed it to 'search'. However, if report was already set, there was no way to unset it. The change here is to make 'report' basically 'search-dryrun', so that one or the other can be set. The other change here is that report would actually exit disabled if it did not find a datasource and notfound=disabled. That was unexpected and would turn cloud-init off, which is not what we wanted. Additionally, consistently use 'enabled' or 'disabled' versus 'enable' and 'disable'. LP: #1669949
2017-03-03ds-identify: move default setting for Ec2/strict_id to a global.Scott Moser
Rather than having the dscheck_Ec2 just know the setting, move it up to a more formal declaration. This will make it look more clean when a distro carries a patch to change it to warn.
2017-03-03ds-identify: record not found in cloud.cfg and always add None.Scott Moser
On a 'not found' result, was not being written at all. That had the unintended effect of '--force' not working. Now, on a 'not found' result: - if reporting: write the list as found (with just 'None'). - if not reporting: only report that there was nothing found. this means that the warning cloud-init will write about ds-identify failing to find a datasource will be written, but cloud-init will still search its fully configured list.
2017-03-03tools/ds-identify: make report mode write namespaced results.Scott Moser
Now, when ds-identify runs in report mode, it still writes to /run/cloud-init.cfg as search does, but it will namespace the result under the top level 'di_report' entry.
2017-03-02Add profile.d script for showing warnings on login.Scott Moser
Z99-cloudinit-warnings.sh can be dropped into /etc/profile.d. Warnings that are written to /var/lib/cloud/instance/warnings will be displayed to the user on stderr when they log in.
2017-03-02Z99-cloud-locale-test.sh: install and make consistent.Scott Moser
Modify upstream packaging to install this file, which was already installed in ubuntu packaging. Also, white space changes from tabs to spaces. Very few things in cloud-init are tabs now. Lastly, remove the executable bit on this as ait is not necessary. Scripts in /etc/profile.d do not have executable bit.
2017-02-28tools/ds-identify: look at cloud.cfg when looking for ec2 strict_id.Scott Moser
In the interest of speed I had skipped the parsing of /etc/cloud/cloud.cfg for the ec2 strict_id setting. In hindsight it seems reasonable for people to put settings there.
2017-02-28tools/ds-identify: disable vmware_guest_customization by default.Scott Moser
ovf_vmware_guest_customization defaults to true in cloud-init, meaning that such customization is disabled. We just missed a return value causing ovf_vmware_guest_customization to effectively default to on. Also, when looking for setting look at /etc/cloud/cloud.cfg. This had been omitted in interest of performance, but we should be looking there.
2017-02-24tools/ds-identify: ovf identify vmware guest customization.Scott Moser
cloud-init by default sets 'disable_vmware_customization' to True. So in ds-identify, we will enable the ovf datasource if: - virt is vmware - 'libdeployPkgPlugin.so' exists as installed by vmware-tools or open-vm-tools. - disable_vmware_customization is configured to True
2017-02-24Identify Brightbox as an Ec2 datasource user.Scott Moser
Brightbox will identify their platform to the guest by setting the product serial to a string that ends with 'brightbox.com'. LP: #1661693
2017-02-24DatasourceEc2: add warning message when not on AWS.Scott Moser
Based on the setting Datasource/Ec2/strict_id, the datasource will now warn once per instance.
2017-02-24ds-identify: add reading of datasource/Ec2/strict_idScott Moser
ds-identify will now read this setting, and thus allow the user to modify ds-identifies behavior via either: 1. builtin setting here cloud-init/ds-identify builtin 2. ds-identify config (/etc/cloud/ds-identify.cfg) 3. system config (/etc/cloud/cloud.cfg.d/*Ec2*.cfg) 4. kernel command line (ci.datasource.ec2.strict_id=true)
2017-02-24tools/ds-identify: add support for found or maybe contributing config.Scott Moser
A check function that returns found or maybe can also now return config that will be written to the resultant /run/cloud.cfg. They do so by setting the variable _RET_excfg.
2017-02-24tools/ds-identify: read the seed directory on Ec2Scott Moser
This just adds checking of the Ec2 seed directory.
2017-02-24tools/ds-identify: use quotes in local declarations.Scott Moser
The following can have cause issue: FOO="bar ; wark" showit() { local b=$FOO echo $b } 4: local: ;: bad variable name The answer is just to use more quotes.
2017-02-24tools/ds-identify: fix documentation of policy setting in a comment.Scott Moser
Just remove some examples that are no longer valid.
2017-02-17ds-identify: only run once per boot unless --force is given.Scott Moser
This makes ds-identify run only once. Previously it would run multiple times each boot as the generator would run more than once. This is potentially dangerous, in that running again might find more attached disks. However that is really only a "lucky" fix if it happens to result differently than the first run. Additionally, we now log the uptime that we started and ended at.
2017-02-10ds-identify: fix checking for filesystem labelScott Moser
has_fs_with_label regressed when refactoring to not have leading and trailing , in DI_FS_LABELS. LP: #1663735
2017-02-10ds-identify: read ds=nocloud properlyScott Moser
The nocloud datasource specifically would look for ds=nocloud or ds=nocloud-net (often augmented with 'seedfrom') on the kernel command line. Fix to return DS_FOUND in that case. LP: #1663723
2017-02-09support nova-lxd by reading platform from environment of pid 1.Scott Moser
Nova lxd will now put the environment variable 'platform' into pid 1's environment to the value 'OpenStack Nova', which is the same as you would find in kvm guests. LP: #1661797
2017-02-09ds-identify: change aarch64 to use the default for non-dmi systems.Scott Moser
aarch64 does support dmi, but OpenStack does not populate guests with this information, and there are currently bugs in qemu preventing it from working correctly see bug #1663304 for more information. So, for the time being, pretend as if there is no dmi data on aarch64, which will make it enable cloud-init even when no datasources are found.
2017-02-05code-style: make master pass pycodestyle (2.3.1) cleanly, currently:Joshua Powers
$ pycodestyle cloudinit/ tests/ tools/ tools/make-mime.py:25:5: E722 do not use bare except' tools/mock-meta.py:252:17: E722 do not use bare except' For tools/make-mime.py:25:5: E722 do not use bare except' the use case is when someone runs ./make-mime.py --attach commis instead of ./make-mime.py --attach commissaire.txt:x-commissaire-host The split can cause a ValueError potentially if there is no: For tools/mock-meta.py:262:17: E722 do not use bare except' the use case is a dictionary look up occurs potentially when an unknown key is given: key_name = key_ids[key_id] Do note that version 2.3.0 falsely reported a dozen or so E302 and E306 errors.
2017-02-03Add tools/ds-identify to identify datasources available.Scott Moser
ds-identify is run here from the generator. If ds-identify does not see any datasources, it can completely disable cloud-init. The big value in this is that if there is no datasource, no python will ever be loaded, and cloud-init will be disabled.o The default policy being added here is: search,found=all,maybe=all,notfound=disabled That means: - enable (in 'datasource_list') all sources that are found. - if none are found, enable all 'maybe'. - if no maybe are found, then disable cloud-init. On platforms without DMI (everything except for aarch64 and x86), the default 'notfound' setting is 'enabled'. This is because many of the detection mechanisms rely on dmi data, which is present only on x86 and aarch64.
2017-01-25tools/mock-meta: support python2 or python3 and ipv6 in both.Scott Moser
Fix mock-meta to work with python2 or python3. Additionally, it will now listen to ipv6 connections, where previously it would only work with ipv4.
2017-01-23build: fix running Make on a branch with tags other than masterScott Moser
running 'make' on a git branch other than master would fail with complaint that the tools/read-version reported a different version than the code. Change to only consider tags starting with 0-9 in read-version.
2017-01-11validate-yaml: use python rather than explicitly python3Scott Moser
The change here is to use '/usr/bin/env python' in validate-yaml.py as all other tools/*.py do. Additionally, change the Makefile to invoke validate-yaml.py with the python that it has selected for other things (PYVER).
2016-12-22LICENSE: Allow dual licensing GPL-3 or Apache 2.0Jon Grimm
This has been a recurring ask and we had initially just made the change to the cloud-init 2.0 codebase. As the current thinking is we'll just continue to enhance the current codebase, its desirable to relicense to match what we'd intended as part of the 2.0 plan here. - put a brief description of license in LICENSE file - put full license versions in LICENSE-GPLv3 and LICENSE-Apache2.0 - simplify the per-file header to reference LICENSE - tox: ignore H102 (Apache License Header check) Add license header to files that ship. Reformat headers, make sure everything has vi: at end of file. Non-shipping files do not need the copyright header, but at the moment tests/ have it.
2016-12-02fix decoding of utf-8 chars in yaml testScott Moser
Python 3 would fail to load yaml from doc/examples/cloud-config-apt.txt when the LANG (specifically LC_CTYPE) was 'C'. The changes here do 2 things: a.) remove the non-ascii characters from the yaml file. b.) fix the validate-yaml.py program to decode using utf-8 specifically rather than using the inherited settings. This fixes it now for ascii and in the future also should non-ascii slip in.
2016-11-10pep8: fix style errors reported by pycodestyle 2.1.0Scott Moser
pycodestyle 2.1.0 is in Ubuntu zesty, and complained about the changes made here. Simple style changes. This makes 'make pep8' pass again when built in a zesty build system with proposed enabled.
2016-08-22azure dhclient-hook cleanupsScott Moser
This adds some function to the generator to maintain the presense of a flag file '/run/cloud-init/enabled' indicating that cloud-init is enabled. Then, only run the dhclient hooks if on Azure and cloud-init is enabled. The test for is_azure currently only checks to see that the board vendor is Microsoft, not actually that we are on azure. Running should not be harmful anywhere, other than slowing down dhclient. The value of this additional code is that then dhclient having run does not task the system with the load of cloud-init. Additionally, some changes to config are done here. * rename 'dhclient_leases' to 'dhclient_lease_file' * move that to the datasource config (datasource/Azure/dhclient_lease_file) Also, it removes the config in config/cloud.cfg that set agent_command to __builtin__. This means that by default cloud-init still needs the agent installed. The suggested follow-on improvement is to use __builtin__ if there is no walinux-agent installed.
2016-08-15Get Azure endpoint server from DHCP clientBrent Baude
It is more efficient and cross-distribution safe to use the hooks function from dhclient to obtain the Azure endpoint server (DHCP option 245). This is done by providing shell scritps that are called by the hooks infrastructure of both dhclient and NetworkManager. The hooks then invoke 'cloud-init dhclient-hook' that maintains json data with the dhclient options in /run/cloud-init/dhclient.hooks/<interface>.json . The azure helper then pulls the value from /run/cloud-init/dhclient.hooks/<interface>.json file(s). If that file does not exist or the value is not present, it will then fall back to the original method of scraping the dhcp client lease file.
2016-08-10tools/read-version: update to address change in versionScott Moser
commit 48ec60ae changed over several tools to use X.Y.Z-XXX-gHASH but missed tools/read-version. The end result was that check_version failed.
2016-08-09make-tarball: older versions of git with --format=tar.Scott Moser
Some older versions of git (Centos 6) do not have --format=tar.gz. To work around this, create a .tar file and then compress it.
2016-08-09read-version: do not attempt git-describe if no git.Scott Moser
Even if there is a .git directory, we can't use git if there is no git executable in the path. In that case just fall back to the cloud-init version.
2016-08-08For upstream snapshot versions do not modify git-describe output.Scott Moser
For upstream version directly use the output of git-describe (X.Y.Z-number.gHASH) rather than rather than changing it to (X.Y.Z+number.gHASH). The rpm version does not allow '-' in Version, so we create and use rpm_upstream_version in the rpm spec file. That is of format: X.Y.Z+number.gHASH
2016-08-05drop modification of version during make-tarball, tools changes.Scott Moser
Modification of the tarball became problematic, as it meant that any tool extracting source would find the orig source tarball different. I found this unusable when trying to use 'gbp buildpackage'. Other changes here are to better support using python3 or python2 for the build. Makefile will try to call the right python version and can be told which python to use. read-version: by adding 'tiny_p' and avoiding the import of cloudinit.util, we need less dependencies to run this.
2016-08-05adjust tools and version information.Scott Moser
upstream snapshots are versioned in the format 'X.Y.Z+<distance>.g<commit>' where X.Y.Z are major, minor, and micro. Distance is number of commits since last annotated tag, and commit is the git commit. bddeb and brpm will now create and use the "upstream version" like above. Things changed here: - tools/make-tarball update cloudinit/version.py to contain the full version support --output support '--long' to always create the long format version string. - bddeb: - use quilt debian source format - use read-version and long version in changelog. - brpm: - change to use read-version and upstream long version in the spec. - flake8 changes - tools/read-version - read version from git or from cloudinit/version. - provide --json output with more nicely formed data.
2016-08-03Update build tools to work with gitLars Kellogg-Stedman
- Update HACKING.rst to include git instructions - update MANIFEST.in and .gitignore to ignore git-related things - replaced tarball generation scripts with git-based script - have the spec files correctly identify themselves as cheetah templates - make brpm work with git
2016-06-15Remove /bin from run-pyflakesJoshua Harlow
2016-06-15Fix a few tools and tests for newer pep8Joshua Harlow