summaryrefslogtreecommitdiff
path: root/cloudinit/util.py
AgeCommit message (Collapse)Author
2014-02-24fixes from testing, force symlinkScott Moser
2014-02-07Remerged with trunkJoshua Harlow
2014-02-01Add a openstack specific datasourceJoshua Harlow
Openstack has a unique derivative datasource that is gaining usage. Previously the config drive datasource provided part of this functionality as well as the ec2 datasource, but since new functionality is being added to openstack is seems benefical to combine the used parts into one datasource just made for handling openstack deployments. This patch factors out the common logic shared between the config drive and the openstack metadata datasource and places that in a shared helper file and then creates a new openstack datasource that readers from the openstack metadata service and refactors the config drive datasource to use this common logic.
2014-01-29merge from trunkScott Moser
2014-01-28cloudinit/util.py: fix pylint complaintsScott Moser
2014-01-28Allow zeros when detecting IPv4 address (e.g. 192.168.0.1)Vlastimil Holer
2014-01-27add 'pathprefix2dict' utility for use by DataSourceNoCloudScott Moser
2014-01-24read_file_or_url: raise UrlError with 404 on ENOENTScott Moser
This makes it easier to call read_file_or_url and handle file or url errors. Now read_file_or_url will raise a UrlError in either case on errors.
2014-01-24pylint and long line fixes.Scott Moser
This fixes up many long lines to be < 80 chars and some other pylint issues. pylint 1.1 (in trusty) is now complaining about the lazy logging, so I'll clean that up when I touch things.
2014-01-23Skip retry and continued fetch of userdata when NOT_FOUNDJoshua Harlow
When a 404 http code comes back from the fetching of ec2 data, instead of retrying immediatly stop the fetching process and in the userdata fetching function handle this case as a special case of no userdata being fetched (an empty string in this case).
2014-01-23fix util.which if PATH is not in environmentScott Moser
This fixes a test case that failed because PATH was unset in the os.environ.
2014-01-23fix broken consumption of /proc/mountsScott Moser
this was checking that the lenghth of the mount line from /proc/mounts was 6, not the number of tokens.
2014-01-23merge from trunkScott Moser
2014-01-15replace get_nested_option_as_list with get_cfg_by_path, improve rupartsScott Moser
this makes runparts take exe_prefix and do string to list conversion inside. that means we don't have to do it in cc_scripts_vendor. Also, get_nested_option_as_list was essentially get_cfg_by_path anyway.
2014-01-09Merge with upstream sourcesBen Howard
2014-01-08Significant re-working of the userdata handling and introduction ofBen Howard
vendordata. Vendordata is a datasource provided userdata-like blob that is parsed similiarly to userdata, execept at the user's pleasure. cloudinit/config/cc_scripts_vendor.py: added vendor script cloud config cloudinit/config/cc_vendor_scripts_per_boot.py: added vendor per boot cloud config cloudinit/config/cc_vendor_scripts_per_instance.py: added vendor per instance vendor cloud config cloudinit/config/cc_vendor_scripts_per_once.py: added per once vendor cloud config script doc/examples/cloud-config-vendor-data.txt: documentation of vendor-data examples doc/vendordata.txt: documentation of vendordata for vendors (RENAMED) tests/unittests/test_userdata.py => tests/unittests/test_userdata.py TO: tests/unittests/test_userdata.py => tests/unittests/test_data.py: userdata test cases are not expanded to confirm superiority over vendor data. bin/cloud-init: change instances of 'consume_userdata' to 'consume_data' cloudinit/handlers/cloud_config.py: Added vendor script handling to default cloud-config modules cloudinit/handlers/shell_script.py: Added ability to change the path key to support vendor provided 'vendor-scripts'. Defaults to 'script'. cloudinit/helpers.py: - Changed ConfigMerger to include handling of vendordata. - Changed helpers to include paths for vendordata. cloudinit/sources/__init__.py: Added functions for helping vendordata - get_vendordata_raw(): returns vendordata unprocessed - get_vendordata(): returns vendordata through userdata processor - has_vendordata(): indicator if vendordata is present - consume_vendordata(): datasource directive for indicating explict user approval of vendordata consumption. Defaults to 'false' cloudinit/stages.py: Re-jiggered for handling of vendordata - _initial_subdirs(): added vendor script definition - update(): added self._store_vendordata() - [ADDED] _store_vendordata(): store vendordata - _get_default_handlers(): modified to allow for filtering which handlers will run against vendordata - [ADDED] _do_handlers(): moved logic from consume_userdata to _do_handlers(). This allows _consume_vendordata() and _consume_userdata() to use the same code path. - [RENAMED] consume_userdata() to _consume_userdata() - [ADDED] _consume_vendordata() for handling vendordata - run after userdata to get user cloud-config - uses ConfigMerger to get the configuration from the instance perspective about whether or not to use vendordata - [ADDED] consume_data() to call _consume_{user,vendor}data cloudinit/util.py: - [ADDED] get_nested_option_as_list() used by cc_vendor* for getting a nested value from a dict and returned as a list - runparts(): added 'exe_prefix' for running exe with a prefix, used by cc_vendor* config/cloud.cfg: Added vendor script execution as default tests/unittests/test_runs/test_merge_run.py: changed consume_userdata() to consume_data() tests/unittests/test_runs/test_simple_run.py: changed consume_userdata() to consume_data()
2013-12-14fix: Log the used method as well.Harm Weites
2013-12-13Cast file path to string.James Slagle
Before passing a path into selinux.matchpathcon, it needs to be casted to a string, since the path could be unicode and selinux.matchpathcon does not support unicode. Closes-Bug: #1260072 LP: #1260072
2013-12-06new: FreeBSD module to support cloud-init on the FBSD10 platform. In itsHarm Weites
current form its still missing some modules though. Supported: -SSH-keys -growpart -growfs -adduser -powerstate
2013-11-13Fix bug lp:1248625 by reading /etc/mtab when mountinfo is not present.Tim Daly Jr.
2013-10-04try to support ephemeral0.1 in cc_mountsScott Moser
2013-10-04simplifications and some function renamesScott Moser
2013-10-03Make {pep8,pylint,test} pass commit.Ben Howard
2013-10-03Moved ephemeralX.Y handling from Datasource into the cc_disk_setup, which ↵Ben Howard
makes it cloud agnostic.
2013-10-02Added ability to define disks via 'ephemeralX.Y'.Ben Howard
Modified cc_mounts to identify whether ephermalX is partitioned. Changed datasources for Azure and SmartOS to use 'ephemeralX.Y' format. Added disk remove functionally
2013-09-27pep8 fixScott Moser
2013-09-25multi_log: only write to /dev/console if it exists.Scott Moser
Some containers lack /dev/console, so when multi_log attempts to open that device and write to it directly things can start going haywire. Here we address this problem by sending console-bound output to stdout and letting init take care of getting it to the console instead. We already configure upstart with "console output", so we need only change systemd to use "journal+console". The one reason that 'console output' might not be sufficient is if the user redirected output with 'output'. Ie: output: init: "> /var/log/my-cloud-init.log" Would then mean all output would go there, and anything that *needed* to go to the console (and was explicitly using multi_log for that purpose) would not get there.
2013-09-25instead of just writing to stdout, write to stdout if no /dev/consoleScott Moser
2013-09-20When selinux is completely disabled functions like restorecon raiseGarrett Holmstrom
exceptions, causing nasty things to happen on instances that boot with selinux=0. The fix is easy: simply consult is_selinux_enabled() first.
2013-09-20Some containers lack /dev/console, so when multi_log attempts to openGarrett Holmstrom
that device and write to it directly things can start going haywire. Here we address this problem by sending console-bound output to stdout and letting init take care of getting it to the console instead. We already configure upstart with "console output", so we need only change systemd to use "journal+console".
2013-09-11merge from trunkScott Moser
2013-09-07Ensure udelta is validJoshua Harlow
It appears that udelta could have been left undefined or left defined as a string "N/A" and then put threw a float formatter previously. Fix that by ensure its set to a default and put strong checking to make sure it is a float before using float formatting.
2013-09-05Merged trunk lp:cloud-initVlastimil Holer
2013-07-30set reading /proc/uptime to false by default.Scott Moser
reading /proc/uptime is going to be slower, and no reason to do it on most things. Better to only do it when you suspect maybe a need for it.
2013-07-30add util.log_time helperScott Moser
The reason for this is that more and more things I was wanting to be able to see how long they took. This puts that time logic into a single place. It also supports (by default) reading from /proc/uptime as the timing mechanism. While that is almost certainly slower than time.time(), it does give millisecond granularity and is not affected by 'ntpdate' having run in between the two events.
2013-07-24add SmartOS datasource supporting Joyent cloudBen Howard
This adds a datasource designed to work on Joyent cloud (SmartOS).
2013-07-23Changed get_serial to be fully parameterized and return the serialBen Howard
initialized. Added a mapping of attributes between cloud-init and smartos.
2013-07-21Use a util helper to do prefix/suffix removal.Joshua Harlow
2013-07-18Added SmartOS datasource and unit tests.Ben Howard
2013-06-19fix and cleanup usage of util.logexcJuerg Haefliger
2013-05-10Fixed merging capabilities.Scott Moser
Instead of previously having merging which was not backwards compatible with the 0.7.1 and prior methods this patch works to ensure said backwards compatible while at the same time making the new merging functionality work in a more customizable manner.
2013-04-25re-work maas datasource and headers call backsScott Moser
Couple of things here: * do not re-try on user-data (404 means 'not here') * re-generate headers on retry requests LP: #1172742
2013-04-17Altering the order of merging.Joshua Harlow
2013-03-26compatibility fixes for Fedora and RHELGreg Padgett
This patch fixes issues in Fedora 18 (and upcoming RHEL 7) which are present due to their use of systemd: - store locale configuration in /etc/locale.conf - store hostname in /etc/hostname - use a symlink for /etc/localtime (prior code would set the timezone but corrupt data in /usr/share/zoneinfo due to presence of symlink) It also contains fixes for issues unrelated to systemd adoption: - explicitly scan /dev/sr0 with blkid in order to get the optical drive in the blkid cache. This prevents an issue on systems running 2.6 kernels (such as RHEL 6) in which config disks on some devices won't be detected unless the device has previously been queried. (For reference, see https://patchwork.kernel.org/patch/1770241/) - append a newline when rewriting sysconfig files, as this is customary text configuration file formatting and is expected by some parsers (such as the ifcfg-rh plugin for NetworkManager)
2013-03-19appease pylint and pep8Scott Moser
* cloudinit/distros/parsers/resolv_conf.py added some pylint overrides with 'plXXXXX' syntax. example: # pl51222 pylint: disable=E0102 The pl51222 there means: http://www.logilab.org/ticket/51222 This specific issue is present in 12.04 pylint, but not 13.04. * pylint doesn't like the requests special handling we have. which makes sense as it is only checking versus one specific version. * general pep8 and pylint cleanups.
2013-03-19merge from trunk rev 800Scott Moser
2013-03-19pylint fixesScott Moser
a.) appease pylint on raring, as it doesn't like subprocess pylint: 0.26.0-1ubuntu1 This is mentioned in comments at http://www.logilab.org/ticket/46273 b.) tests/unittests/test_util.py: the mountinfo lines are longer than 80 chars. Just disable long lines complaints for this file.
2013-03-19merge from trunk at revno 799Scott Moser
2013-03-13fix / workaround potential for socket.getaddrinfo to raise socket.errorScott Moser
As reported in bug 1154599, I'm seeing this on my desktop system: $ python -c \ 'from cloudinit import util; print util.is_resolvable("brickies.neiit")' Traceback (most recent call last): File "<string>", line 1, in <module> File "cloudinit/util.py", line 865, in is_resolvable socket.SOCK_STREAM, socket.AI_CANONNAME) LP: #1154599
2013-03-10util.parse_mount_info(): handle short lines.Blair Zajac
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-14 23:25:18 +0000