| Age | Commit message (Collapse) | Author |
|---|
|
On python 2.7 and earlier (CentOS 6 & 7), UrlErrors raised by requests do
not report the url which failed. In such cases, append the url if not
present in the error message.
This fixes nightly CI failures at
https://jenkins.ubuntu.com/server/view/cloud-init/.
|
|
Without this the entire stage can fail, which will leave an instance
unaccessible.
Reviewed-by: Tom Kirchner <tjk@amazon.com>
Reviewed-by: Matt Nierzwicki <nierzwic@amazon.com>
Reviewed-by: Ben Cressey <bcressey@amazon.com>
|
|
This will change all instances of LOG.warn to LOG.warning as warn
is now a deprecated method. It will also make sure any logging
uses lazy logging by passing string format arguments as function
parameters.
|
|
This has been a recurring ask and we had initially just made the change to
the cloud-init 2.0 codebase. As the current thinking is we'll just
continue to enhance the current codebase, its desirable to relicense to
match what we'd intended as part of the 2.0 plan here.
- put a brief description of license in LICENSE file
- put full license versions in LICENSE-GPLv3 and LICENSE-Apache2.0
- simplify the per-file header to reference LICENSE
- tox: ignore H102 (Apache License Header check)
Add license header to files that ship.
Reformat headers, make sure everything has vi: at end of file.
Non-shipping files do not need the copyright header,
but at the moment tests/ have it.
|
|
when user-data was not decodable, cloud-init would raise exception.
LP: #1532072
|
|
|
|
handling of cloud-config-archive input would fail in fully_decoded_payload.
part.get_charset() would return a Charset object, but
get_charset.input_codec is a string suitable for passing to decode.
This handles that correctly, and is more careful about binary data inside
input.
The test added verifies that cloud-config inside a cloud-config-archive
is handled correctly and also that binary data there is ignored without
exceptions raised.
LP: #1445143
|
|
This is to work-around the fact that email.message_from_string uses
cStringIO in Python 2.6, which can't handle Unicode.
|
|
|
|
an include can include a gzip type also or binary data
so avoid conversion here.
|
|
just seems to make more sense to decode here.
Add a test showing the previous failure (testBytesInPayload)
And one that should pass (testStringInPayload)
Also, add a test for unencoded content in the ovf xml (test_userdata_plain)
And explicitly set encoding on another test (test_userdata_found).
|
|
base64 encoded data.
|
|
|
|
- Refactor "fully" decoding the payload of a text/* part. In Python 3,
decode=True only means to decode according to Content-Transfer-Encoding, not
according to any charset in the Content-Type header. So do that.
|
|
- Remove str() wrappers to second argument to write_files() where it is no
longer necessary.
Also: Fixed a couple of other octal literals which clearly weren't being
tested.
|
|
|
|
|
|
to be behind trunk.
`tox -e py27` passes full test suite. Now to work on replacing mocker.
|
|
This fixes up many long lines to be < 80 chars and some other
pylint issues. pylint 1.1 (in trusty) is now complaining about
the lazy logging, so I'll clean that up when I touch things.
|
|
vendordata.
Vendordata is a datasource provided userdata-like blob that is parsed
similiarly to userdata, execept at the user's pleasure.
cloudinit/config/cc_scripts_vendor.py: added vendor script cloud config
cloudinit/config/cc_vendor_scripts_per_boot.py: added vendor per boot
cloud config
cloudinit/config/cc_vendor_scripts_per_instance.py: added vendor per
instance vendor cloud config
cloudinit/config/cc_vendor_scripts_per_once.py: added per once vendor
cloud config script
doc/examples/cloud-config-vendor-data.txt: documentation of vendor-data
examples
doc/vendordata.txt: documentation of vendordata for vendors
(RENAMED) tests/unittests/test_userdata.py => tests/unittests/test_userdata.py
TO: tests/unittests/test_userdata.py => tests/unittests/test_data.py:
userdata test cases are not expanded to confirm superiority over vendor
data.
bin/cloud-init: change instances of 'consume_userdata' to 'consume_data'
cloudinit/handlers/cloud_config.py: Added vendor script handling to default
cloud-config modules
cloudinit/handlers/shell_script.py: Added ability to change the path key to
support vendor provided 'vendor-scripts'. Defaults to 'script'.
cloudinit/helpers.py:
- Changed ConfigMerger to include handling of vendordata.
- Changed helpers to include paths for vendordata.
cloudinit/sources/__init__.py: Added functions for helping vendordata
- get_vendordata_raw(): returns vendordata unprocessed
- get_vendordata(): returns vendordata through userdata processor
- has_vendordata(): indicator if vendordata is present
- consume_vendordata(): datasource directive for indicating explict
user approval of vendordata consumption. Defaults to 'false'
cloudinit/stages.py: Re-jiggered for handling of vendordata
- _initial_subdirs(): added vendor script definition
- update(): added self._store_vendordata()
- [ADDED] _store_vendordata(): store vendordata
- _get_default_handlers(): modified to allow for filtering
which handlers will run against vendordata
- [ADDED] _do_handlers(): moved logic from consume_userdata
to _do_handlers(). This allows _consume_vendordata() and
_consume_userdata() to use the same code path.
- [RENAMED] consume_userdata() to _consume_userdata()
- [ADDED] _consume_vendordata() for handling vendordata
- run after userdata to get user cloud-config
- uses ConfigMerger to get the configuration from the
instance perspective about whether or not to use
vendordata
- [ADDED] consume_data() to call _consume_{user,vendor}data
cloudinit/util.py:
- [ADDED] get_nested_option_as_list() used by cc_vendor* for
getting a nested value from a dict and returned as a list
- runparts(): added 'exe_prefix' for running exe with a prefix,
used by cc_vendor*
config/cloud.cfg: Added vendor script execution as default
tests/unittests/test_runs/test_merge_run.py: changed consume_userdata() to
consume_data()
tests/unittests/test_runs/test_simple_run.py: changed consume_userdata() to
consume_data()
|
|
|
|
|
|
|
|
|
|
Instead of being restricted to only gzip
compressing the overall mime segment or
individual included segments, allow for
each mime segment to be gzip compressed.
LP: #1203203
|
|
|
|
|
|
|
|
LP: #1065116
|
|
its original content type said it is, make sure we set
the new value, also unsure if the old top level message
should have the same header (which will flip-flop).
|
|
indexes (since they will be handled beforehand) and fix
the types being checked on the root of the archive format
to be a tuple instead of a list (which oddly causes complaints).
|
|
variable has a little more meaning and by default look in
metadata for 'launch-index' and have ec2 instead look for
a different variable (thus allowing more datasources to just work).
|
|
'launch-index' key that we copy that key over to the right
header (which will then be used later when assigning the
'real' header when the message is attached)
|
|
before we look into the payload as well as make the skip
test a function that the datasource module can also use.
|
|
userdata based on a launch-index (or leave userdata
alone if none is provided by the datasource). This
works by doing the following.
1. Adjusting the userdata processor to attempt to
inject a "Launch-Index" header into the messages
headers (by either taking a header that already exists
or by looking into the payload to see if it exists
there).
2. Adjust the get_userdata ds function to apply a filter
on the returned userdata (defaulting to false) that
will now use the datasources get_launch_index value
to restrict the 'final' message used in consuming
user data (the same behavior if not existent).
3. Further down the line processes that use the 'resultant'
userdata now will only see the ones for there own launch
index (ie cloud-config will be restricted automatically
and so on) and are unaffected (although they can now
ask the cloud object or the datasource for its launch index
via the above new ds method.
|
|
make pep8 now is silent on precise's pep8 ( 0.6.1-2ubuntu2).
|
|
other code to have user/group parsing in util instead
of in stages.py, renames decomp_str to decomp_gzip since
it is more meaningful when named that (as thats all it can
decompress).
|
|
2. Fix pylint warning in userdata about unused variable
|
|
easily
|
|
|
|
turn it back off, if one of those is encountered
|
|
a. This allows for more properties to be added as needed in the future, instead of being very restrictive.
2. Fix up all uses of the url reading to now use this new response object.
3. Also fixup user data including, such that if no response actual occurs the url content is not further processed.
|
|
|
|
for case insensitive include statements to be used
|
|
being in here.
This class will now just contain user data parsing, leaving the handler running to happen elsewhere.
|
|
appropriate annotations + metaclasses.
|
|
2. Add logging of unknown content types + seperate payload logging detail message into a separate function.
|
|
|
|
|
|
for now it seems to make organizational sense to put it here.
|
