summaryrefslogtreecommitdiff
path: root/cloudinit/util.py
AgeCommit message (Collapse)Author
2017-03-29support resizing partition and rootfs on system booted without initramfs.Steve Langasek
When booted without an initramfs, the root device will be /dev/root, not a named device. There is partial support for this when resizing filesystems, but not for growing partitions, without which it doesn't do much good. Move the /dev/root resolution code to util.py and use it from cc_growpart.py. Also, booting without an initramfs only works with a root= argument that's either a kernel device name (which is unstable) or a partition UUID. Handle the case of root=PARTUUID=value, not just LABEL and UUID. LP: #1677376
2017-03-20cloudinit.net: add network config v2 parsing and renderingRyan Harper
Network configuration version 2 format is implemented in a package called netplan (nplan)[1] which allows consolidated network config for multiple network controllers. - Add a new netplan renderer - Update default policy, placing eni and sysconfig first This requires explicit policy to enable netplan over eni on systems which have both (Yakkety, Zesty, UC16) - Allow any network state (parsed from any format cloud-init supports) to render to v2 if system supports netplan. - Move eni's _subnet_is_ipv6 to common code for use by other renderers - Make sysconfig renderer always emit /etc/syconfig/network configuration - Update cloud-init.service systemd unit to also wait on systemd-networkd-wait-online.service 1. https://lists.ubuntu.com/archives/ubuntu-devel/2016-July/039464.html
2017-03-17net: add renderers for automatically selecting the renderer.Scott Moser
Previously, the distro had hard coded which network renderer it would use. This adds support for just picking the right renderer based on what is available. Now, that can be set via a priority in system_info, but should generally work. That config looks like: system_info: network: renderers: ["eni", "sysconfig"] When no renderers are found, a specific RendererNotFoundError is raised. stages.py is modified to catch that and log it at error level. This path should not really be exercised, but could occur if for example an Ubuntu system did not have ifupdown, or a rhel system did not have sysconfig. In such a system previously we would have quietly rendered ENI configuration but that would have been ignored. This is one step better in that we at least log the error.
2017-01-11Get early logging logged, including failures of cmdline url.Scott Moser
Failures to load the kernel command line's url (cloud-config-url=) would previously get swallowed. This should make it much more obvious when that happens. With logging going to expected places at sane levels (WARN will go to stderr by default).
2016-12-22LICENSE: Allow dual licensing GPL-3 or Apache 2.0Jon Grimm
This has been a recurring ask and we had initially just made the change to the cloud-init 2.0 codebase. As the current thinking is we'll just continue to enhance the current codebase, its desirable to relicense to match what we'd intended as part of the 2.0 plan here. - put a brief description of license in LICENSE file - put full license versions in LICENSE-GPLv3 and LICENSE-Apache2.0 - simplify the per-file header to reference LICENSE - tox: ignore H102 (Apache License Header check) Add license header to files that ship. Reformat headers, make sure everything has vi: at end of file. Non-shipping files do not need the copyright header, but at the moment tests/ have it.
2016-12-21Fix config order of precedence, putting kernel command line over system.Wesley Wiedenmeier
The correct order of precedence when reading the base config: builtin config system config kernel command line provided config. This reverts commit 63501f44, which actually broke the behavior it reported to fix. It also adds some unit tests to ensure this behavior is not broken again. LP: #1582323
2016-11-22Improve formatting for ProcessExecutionErrorWesley Wiedenmeier
This replaces long single lines in a log or console output with multiple lines that are much easier to read. It indents the stdout and stderr so logs are more easily read also.
2016-11-10pep8: fix style errors reported by pycodestyle 2.1.0Scott Moser
pycodestyle 2.1.0 is in Ubuntu zesty, and complained about the changes made here. Simple style changes. This makes 'make pep8' pass again when built in a zesty build system with proposed enabled.
2016-11-01dmidecode: Allow dmidecode to be used on aarch64Robert Schweikert
aarch64 systems have functional dmidecode, so allow that to be used. - aarch64 has support for dmidecode as well
2016-10-20Add support for snap create-user on Ubuntu Core images.Ryan Harper
Ubuntu Core images use the `snap create-user` to add users to an Ubuntu Core system. Add support for creating snap users by adding a key to the users dictionary. users: - name: bob snapuser: bob@bobcom.io Or via the 'snappy' dictionary: snappy: email: bob@bobcom.io Users may also create a snap user without contacting the SSO by providing a 'system-user' assertion by importing them into snapd. Additionally, Ubuntu Core systems have a read-only /etc/passwd such that the normal useradd/groupadd commands do not function without an additional flag, '--extrausers', which redirects the pwd to /var/lib/extrausers. Move the system_is_snappy() check from cc_snappy module to util for re-use and then update the Distro class to append '--extrausers' if the system is Ubuntu Core.
2016-10-19Fix python2.6 things found running in centos 6.Scott Moser
This gets the tests running in centos 6. * ProcessExecutionError: remove setting of .message Nothing in cloud-init seems to use .message anywhere, so it does not seem necessary. The reason to change it is that on 2.6 it spits out: cloudinit/util.py:286: DeprecationWarning: BaseException.message * tox.ini: add a centos6 environment the tox versions listed here replicate a centos6 install with packages from EPEL. You will still need a python2.6 to run this env so we do not enable it by default.
2016-09-21Decode unicode types in decode_binaryRobert Schweikert
The test in decode_binary for six.text_type was incorrect as that includes unicode type in Python 2 which should actually be decoded. When the type is string_types we now properly check only for basestring and str in Python 2 and Python 3 respectively and return the given blob without making an attempt to decode.
2016-09-21subp: add 'update_env' argumentScott Moser
In order for a caller to use 'env' argument of subp, they will realistically do: env = os.environ.copy() env['FOO'] = 'BZR' subp(cmd, env=env) This shortens that to be: subp(cmd, update_env={'FOO': 'BZR'}) Add tests, and update growpart tests to use mock when playing with os.environ.
2016-09-15dmidecode: run dmidecode only on i?86 or x86_64 arch.Robert Schweikert
Dmidecode is not going to run successfully on anything other than an x86 or x86_64, just avoid running it anywhere else.
2016-08-25python2.6: fix dict comprehension usage in _lsb_release.Joshua Harlow
This syntax doesn't work in python 2.6
2016-08-22Fix Gentoo net config generationMatthew Thode
This gets Gentoo work on simple configs with static IPs or DHCP on physical interfaces. This gets Gentoo bootable again.
2016-08-18Apt: add new apt configuration formatChristian Ehrhardt
This adds an improved apt configuration format that is fully backwards compatible with previous behavior. This is mostly copied from curtin's implementation. It does: * clean up and centralizes many of the top level 'apt_*' values that previously existed into a single top level 'apt'key. * support a 'source' in apt/sources/entry that has only a key * documents new features and adds tests. See the added doc/examples/cloud-config-apt.txt for more information.
2016-08-12DigitalOcean: use the v1.json endpointBen Howard
Per [1], DigitalOcean provides the metadata in multiple formats. The JSON document is the preferred endpoint. Changes: - Switch to the v1.json meta-data endpoint - Identify droplet identity from SMBIOS - Only poll for metadata when the instance is confirmed to be a droplet - Removal of hard-coded mirrors Additionally, centralize the gates on running 'dmidecode' on arm arches, and update tests to address. [1] https://developers.digitalocean.com/documentation/metadata/
2016-06-15python3 OSError does not have a .messageScott Moser
2016-06-06Rebase against masterJoshua Harlow
2016-05-24rebased with upstream and reolved merge conflictsChristian Ehrhardt
2016-05-23final pep8 check fixupsChristian Ehrhardt
2016-05-23support apt_sources to be a dictionaryChristian Ehrhardt
key is the filename, and "old" input shall be handled as it was all the time. For compatibility this will (continue to) overwrite the file of multiple options that did not specify an output file (they all get the same default). Yet it will process them all - as it always did - e.g. to add the keys of all of them. Any users of the new format won't have these issues, as they will always have a key.
2016-05-19Remerge against head/masterJoshua Harlow
2016-05-19Adjust net module to be more isolatedJoshua Harlow
This allows it to be used outside of cloudinit more easily in the future.
2016-05-12Fix up a ton of flake8 issuesJoshua Harlow
2016-05-11Fix py26 for rhel (and older versions of python)Joshua Harlow
2016-03-24make get_cmdline read /proc/1/cmdline if inside a containerScott Moser
This follows behavior of systemd/cloud-init-generator. This way you can feed a command line into lxc container.
2016-03-14change return value for dmi data of all \xff to be ""Scott Moser
Previously we returned a string of "." the same length as the dmi field. That seems confusing to the user as "." would seem like a valid response when in fact this value should not be considered valid. So now, in this case, return empty string.
2016-03-14strip return of dmidecode and do so before checking for all "."Scott Moser
2016-03-14change where we handle the translationScott Moser
2016-03-14change return value for dmi data of all \xff to be ""Scott Moser
Previously we returned a string of "." the same length as the dmi field. That seems confusing to the user as "." would seem like a valid response when in fact this value should not be considered valid. So now, in this case, return empty string.
2016-03-10dmi data: fix failure of reading dmi data for unset dmi valuesScott Moser
it is not uncommon to find dmi data in /sys full of 'ff'. utf-8 decoding of those would fail, causing warning and stacktrace. Return '.' instead of \xff. This is what dmidecode would return. $ dmidecode --string system-product-name
2016-03-10improve commentScott Moser
2016-03-10dmi data: fix failure of reading dmi data for unset dmi valuesScott Moser
it is not uncommon to find dmi data in /sys full of 'ff'. utf-8 decoding of those would fail, causing warning and stacktrace. Return '.' instead of \xff. This maps to what dmidecode would return $ dmidecode --string system-product-name .................................
2016-03-03Fix pyflake/pyflake3 errorsRyan Harper
Now we can run make check to assess pep8, pyflakes for python2 or 3 And execute unittests via nosetests (2 and 3).
2016-03-03Update pep8 runner and fix pep8 issuesRyan Harper
2016-01-28Use systemd-detect-virt to detect a container.Martin Pitt
running-in-container is an Ubuntu-ism and going away. LP: #1539016
2015-09-01Handle symlink mount points in mount_cb.Daniel Watkins
The Azure data source now uses a /dev/disk symlink to identify devices, but the dereferenced version of this appears in the mount table. mount_cb therefore doesn't identify when a disk is already mounted, and attempts to mount it a second time (which fails with NTFS).
2015-08-06tests passScott Moser
2015-07-27_read_dmi_syspath: fix bad log message causing unintended exceptionScott Moser
2015-05-14read_seeded: fix reed_seeded after regressionScott Moser
read_seeded was assuming a Response object back from load_tfile_or_url but load_tfile_or_url was returning string. since the only other user of this was a test, move load_tfile_or_url to a test, and just do the right thing in read_seeded. LP: #1455233
2015-04-16fix cloud-config-archive handlingScott Moser
handling of cloud-config-archive input would fail in fully_decoded_payload. part.get_charset() would return a Charset object, but get_charset.input_codec is a string suitable for passing to decode. This handles that correctly, and is more careful about binary data inside input. The test added verifies that cloud-config inside a cloud-config-archive is handled correctly and also that binary data there is ignored without exceptions raised. LP: #1445143
2015-03-04Add util.message_from_string to wrap email.message_from_string.Daniel Watkins
This is to work-around the fact that email.message_from_string uses cStringIO in Python 2.6, which can't handle Unicode.
2015-03-04Use more consistent logging invocation.Daniel Watkins
2015-03-04Convert dmidecode values to sysfs names before looking for them.Daniel Watkins
dmidecode and /sys/class/dmi/id/* use different names for the same information. This modified the logic in util.read_dmi_data to map from dmidecode names to sysfs names before looking in sysfs.
2015-03-02get_cmdline_url: fix in python3 when callingScott Moser
get_cmdline_url was passing a string to response.contents.startswith() where response.contents is now bytes. this changes it to convert input to text, and also to default to text.
2015-03-02util.py: remove 'print' debug statementScott Moser
2015-02-25move towards user-data being binaryScott Moser
UrlResponse: biggest change... make readurl return bytes, making user know what to do with it. util: add load_tfile_or_url for loading text file or url as read_file_or_url now returns bytes ec2_utils: all meta-data is text, remove non-obvious string translations DigitalOcean: adjust for ec2_utils DataSourceGCE, DataSourceMAAS: user-data is binary other fields are text. openstack.py: read paths without decoding to text. This is ok as paths other than user-data are json, and load_json will handle load_file still returns text, and that is what most things use.
2015-02-12mount_cb: fix scoping of an exception in python3Scott Moser