summaryrefslogtreecommitdiff
path: root/cloudinit/util.py
AgeCommit message (Collapse)Author
2013-03-07merge from trunkScott Moser
2013-03-07pep8 and pylint fixesScott Moser
2013-03-06Make conf.d and the default merging use the new merging algos.Joshua Harlow
2013-03-06Continue working on merging code.Joshua Harlow
2013-03-03add default log value to get_mount_infoScott Moser
2013-03-01initial stab at growpart moduleScott Moser
LP: #1136936
2013-01-30Support package versions for the generic package config modulectracey
Augmenting the package version support to be available when specifying extra packages to be installed at boot via the 'packages:' yaml key. This change also improves type checking and add a configuration example to the docs.
2013-01-27Adding package versioning logic to package_commandCraig Tracey
This change adds the ability to provide specific package versions to Distro.install_packages and subsequently Distro.package_command. In order to effectively use Distro.install_packages, one is now able to pass a variety of formats in order to easily manage package requirements. These are examples of what can be passed: - "package" - ["package1","package2"] - ("package",) - ("package", "version") - [("package1",)("package2",)] - [("package1", "version1"),("package2","version2")] This change also adds the option to install a specific version for the puppet configuration module. This is especially important here as successful puppet deployments are highly reliant on specific puppet versions.
2013-01-16DataSourceConfigDrive: consider CD rom as valid config-drive source.Scott Moser
previously, there was an attempt in the config drive source to limit the source device to a "full block device" rather than a partition. This was done by a simplistic approach of checking that the last character of the name was not a number. That was filtering out CD-rom devices (sr0). Now, we have a bit more sophisticated approach to that same problem. We filter out block devices that have a 'partition' entry in /sys/class/block/DEVICE_NAME/partition . LP: #1100545
2012-11-12Rebased with HEAD and resolved conflicts.Joshua Harlow
2012-11-10whitespace / indentation cleanupsScott Moser
These changes were pulled out of the previous merge (cc_yum_add_repo) as they were unrelated there. Re-applying them here.
2012-10-23use only util methods for reading/loading/appending/peekingScott Moser
Use only util methods for reading/loading/appending/peeking at files since it is likely soon that we will add a new way of adjusting the root of files read, also it is useful for debugging to track what is being read/written in a central fashion.
2012-10-10System config niceness!Joshua Harlow
1. Move out the old helpers that provided oop access/reading/writing to various standard conf files and place those in parsers instead. 2. Unify the 'update_hostname' which varied very little between distros and make it generic so that subclasses can only provide a couple of functions to obtain the hostname updating functionality 3. Implement that new set of functions in rhel/debian 4. Use the new parsers chop_comment function for similar use cases as well as add a new utils make header function that can be used for configuration files that are newly generated to use (less duplication here of this same thing being done in multiple places. 5. Add in a distro '_apply_hostname' which calls out to the 'hostname' program to set the system hostname (more duplication elimination). 6. Make the 'constant' filenames being written to for configuration by the various distros be instance members instead of string constants 'sprinkled' throughout the code
2012-10-10Merge the resolv.conf branch inJoshua Harlow
preparation for having a new branch for all of these changes.
2012-09-28Sync with head and fix conflicts.Joshua Harlow
2012-09-28Rework the rest of the locations that usedJoshua Harlow
the previous 'user' and make those locations go through the new distros functions to select the default user or the user list (depending on usage). Adjust the tests to check the new 'default' field that signifies the default user + test the new method to extract just the default user from a normalized user dictionary.
2012-09-28add 'safeyaml' to cloudinitScott Moser
In 0.7.0 we started using yaml.safe_load to load data rather than yaml.load. Some producers (namely, ubuntu MAAS created) have produced cloud-config data in the past that included python unicode types. This creates a specialized safe_loader that is basically safe_load + support for python unicode.
2012-09-211. Cleanup the user creation so that the distro class isJoshua Harlow
responsible only for creating users and groups and normalizing a input configuration into a normalized format that splits up the user list, the group list and the default user listsand let the add user/group config module handle calling those methods to add its own users/groups and the default user (if any). 2. Also add in tests for this normalization process to ensure that it is pretty bug free and works with the different types of formats that users/groups/defaults + options can take.
2012-09-20Use only util methods for reading/loading/appending/peekingJoshua Harlow
at files since it is likely soon that we will add a new way of adjusting the root of files read, also it is useful for debugging to track what is being read/written in a central fashion.
2012-09-20Add a resolv.conf parser that can be easilyJoshua Harlow
used for adjusting a resolv.conf formatted file and use this to adjust the resolv.conf in the redhat distro instead of replacing the previous resolv.conf completely.
2012-08-27For the userdata 'post-filtering' add in a new folder thatJoshua Harlow
can contain filters that serve this purpose only and add in the initial launch-index filter and replace the code in the datasource class that previously did this.
2012-08-22fix issues from 'make pylint'Scott Moser
In an effort to pylint errors about NonImlementedError and add_default_user, I moved this method to distro and genericized it. Now, assuming a sane 'create_user' for the distro, this should work. Also: * removed the unused set_configured_user method
2012-08-22fix errors from 'make pep8'Scott Moser
2012-08-22Merge with lp:cloud-initBen Howard
2012-08-22fix pep8 complaints.Scott Moser
make pep8 now is silent on precise's pep8 ( 0.6.1-2ubuntu2).
2012-08-20Dropped hidden command; replaced with logstring. Also changed useradd ↵Ben Howard
command to use log options over short
2012-08-20Added "userless" mode to cloud-init for handling the creation of theBen Howard
users and the default user on Ubuntu. cloudinit/config/cc_users_groups.py: new cloud-config module for creating users and groups on instance initialization. - Creates users and group - Sets "user" directive used in ssh_import_id cloudinit/config/cc_ssh_import_id.py: module will rely upon users_groups for setting the default user. Removed assumption of 'ubuntu' user. cloudinit/distros/__init__.py: Added new abstract methods for getting and creating the default user. cloudinit/distros/ubuntu.py: Defined abstract methods for getting and and creating the default 'ubuntu' user on Ubuntu instances. cloudinit/util.py: Added ability to hide command run through util.subp to prevent the commands from showing in the logs. Used by user_groups cloud-config module. config/cloud.cfg: Removed "user: ubuntu" directive and replaced with new user-less syntax. doc/examples/cloud-config.txt: Documented the creation of users and groups.
2012-08-06add protection against dns-redirection to is_resolvableScott Moser
In an effort to make the EC2 Datasource's search under ec2.archive.ubuntu.com resilient against dns redirection, we add some code to is_resolvable. One future enhancement for this would be to protect against server side round robin results. Ie, if 'bogus-entry' returned 10.0.1.1 one time, and then 10.0.1.2 a second time. We could check if results where within the same 3 octets, and assume invalid if they were.
2012-07-16improve debug via _CLOUD_INIT_SAVE_STDIN and _CLOUD_INIT_SAVE_STDOUTScott Moser
This does 2 things: a.) fixes broken logic in 'close_stdin' previously _CLOUD_INIT_SAVE_STDIN had to be set to false to preserve stdin. "save_stdin" should be true to indicate it should be saved. The net result is that you can stuff just add import pdb; pdb.set_trace() to code, and then run something like: sudo _CLOUD_INIT_SAVE_STDIN=1 _CLOUD_INIT_SAVE_STDOUT=1 \ cloud-init single --name=mounts --frequency=always And enter the debugger even if you had a 'output' that would redirect stdin/out by default, like: output: {all: '| tee -a /var/log/cloud-init-output.log'}
2012-07-12load_yaml: if conversion fails, but string is empty, return defaultScott Moser
the problem sovled here is that most callers of yaml_load do not pass 'None' as a allowed type. I didn't want to change all the callers. If the yaml.safe_load ended up being None we were raising a TypeError and that was getting logged (meaning output to console). Even though the this was not really bad. So, if the type is not in the list, *and* it is not empty, then log exception. If it in the list and empty, just debug. empty input was occurring when cloud-config was empty (no user-data)
2012-07-11do not capture runparts outputScott Moser
runparts output was being captured. This meant that output of user-scripts was being captured and not sent to stdout (console).
2012-07-10Fixes 1012854 by implementing file writing, adjustsJoshua Harlow
other code to have user/group parsing in util instead of in stages.py, renames decomp_str to decomp_gzip since it is more meaningful when named that (as thats all it can decompress).
2012-07-10send welcome message after logging has been appliedScott Moser
in the 'cloud-init init' stages, we want the welcome message to get to the correct output as specified by the system's configuration. Ie, if the local /etc/cloud.config.d had 'output' or 'log_cfg' settings we want those to be able to affect the welcome message also. In normal operation, nothing else will go to stdout or stderr before this, and likely/hopefully nothing terribly important to the logs.
2012-07-09Use the common 'is_false' routine for testing false here insteadharlowja
of a custom list that may confuse people trying to use this.
2012-07-09Update chownbyname to catch the key error when users/groups that are not knownJoshua Harlow
are provided and rethrow it as a OSError (which seems reasonable) and adjust its usage in the log file touching/permission modification stage to catch this error and log it.
2012-07-09fix 'make pylint' warningsScott Moser
On my system (quantal) this 'make pylint' does not complain now.
2012-07-09DataSourceEc2: only do dns check in mirror selectionScott Moser
This returns the check for an archive mirror in the DataSourceEc2 to only do so by DNS resolution. The 'rework' branch had made the check wait and timeout on attempts to reach the mirror. This resulted in 120 seconds of waiting before failure. For now, just go back to the old situation of checking by dns.
2012-07-03Add a check on 'chownbyname' that catches the keyerror and logs it (instead ↵Joshua Harlow
of failing)
2012-07-021. Move the welcome message template string to a constant at the top of the ↵Joshua Harlow
module 2. Fix the usage of multi_log to log to only one of the places (for now) 3. Update comment about multi-log and why write_file isn't used in this case
2012-07-01Merge in pending changes.harlowja
2012-07-011. Rename util functions to is_true and is_falseharlowja
2. Move the config loading functions to where they are used (in stages) 3. Adjust cc_set_passwords to use the is_true and is_false renamed functions 4. Adjust the init stage to have a _read_base_config function used to load the base 'initial' configuration from the following locations a. Kernel cmdline b. Conf.d location (+ the cloud.cfg location) c. Built-in configuration
2012-07-01Add a multi log function that can write to stderr, console and a log debug, ↵Joshua Harlow
useful in certain cases
2012-06-29Refactor the selinux guard to aid in mockingharlowja
1. Adjust the test_util after this mocking to be cleaner
2012-06-29Just log the number of commands 'shellified'Joshua Harlow
2012-06-29Don't always log stdout and stderr (exceptions will capture them if it fails)Joshua Harlow
2012-06-29Reduce the log levels by removing a few statements that happen to often (and ↵Joshua Harlow
aren't very meaningful)
2012-06-29Add the rc code for blkid '2' being valid with a note to as why.Joshua Harlow
2012-06-261. Add a chdir context managerJoshua Harlow
2. Add a abs path joining function that will return the absolute path of a combined path (where applicable)
2012-06-23Fix a bunch of == None casesharlowja
2012-06-221. Move the getkeybyid function back here but add some slight adjustmentsJoshua Harlow
a. Instead of executing a bash string, write out a temporary file and then just execute '/bin/sh' on that file with the right arguments instead. 2. Rename util.SilentTemporaryFile to util.ExtendedTemporaryFile and update the usages of the previous name accordingly, this better reflects what this temp file is. 3. More teenie pep8 line length fixings