| Age | Commit message (Collapse) | Author |
|---|
|
|
|
This would cause problems in the event that we actually had a bad
clock. We add a retry in the main (for test) also, to ensure that
the oauth timestamp fix gets in place.
LP: #1499869
|
|
we want cloud-init user agent to be present even if the user
provided some headers. In the event that they provided User-Agent,
this will respect their wishes.
|
|
the previous version was broken. The vital fixes here are:
* adding parsedate and oauth1 imports to url_helper
* fix skew_data usage intending to use self.skew_data
Additionally:
* reorder imports in url_helper
* fixes to python3 -m cloudinit.sources.DataSourceMaas
LP: #1488507
|
|
|
|
|
|
|
|
|
|
python2 scoping is different and running wait_for_url in python3
results in a use of undeclared variable 'e'.
$ python3 -c 'from cloudinit import url_helper; \
url_helper.wait_for_url("o", max_wait=3,timeout=1, exception_cb=print)'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "cloudinit/url_helper.py", line 358, in wait_for_url
exception_cb(msg=status_msg, exception=e)
|
|
UrlResponse: biggest change... make readurl return bytes, making user
know what to do with it.
util: add load_tfile_or_url for loading text file or url
as read_file_or_url now returns bytes
ec2_utils: all meta-data is text, remove non-obvious string translations
DigitalOcean: adjust for ec2_utils
DataSourceGCE, DataSourceMAAS: user-data is binary other fields are text.
openstack.py: read paths without decoding to text. This is ok as paths
other than user-data are json, and load_json will handle
load_file still returns text, and that is what most things use.
|
|
to be behind trunk.
`tox -e py27` passes full test suite. Now to work on replacing mocker.
|
|
This just removes comments '# pylint:' things and other code
remnents of pylint.
|
|
Fixed all complaints from running "make pep8". Also version locked
pep8 in test-requirements.txt to ensure that pep8 requirements don't
change without an explicit commit.
|
|
Waiting around for a metadata service in a given datasource means that if its
not there all the subsequent datasources have to wait, and boot is slowed down.
As it is right now, EC2 is the only one that has the right to wait. In the
past, we had to wait around for the EC2 metadata service. I really do not want
to extend that courtesy to other cloud platforms. A network based metadata
service should be up as soon as networking is up.
|
|
|
|
|
|
|
|
|
|
Openstack has a unique derivative datasource
that is gaining usage. Previously the config
drive datasource provided part of this functionality
as well as the ec2 datasource, but since new
functionality is being added to openstack is
seems benefical to combine the used parts into
one datasource just made for handling openstack
deployments.
This patch factors out the common logic shared
between the config drive and the openstack
metadata datasource and places that in a shared
helper file and then creates a new openstack
datasource that readers from the openstack metadata
service and refactors the config drive datasource
to use this common logic.
|
|
This makes it easier to call read_file_or_url and handle file or url
errors. Now read_file_or_url will raise a UrlError in either case
on errors.
|
|
|
|
When a 404 http code comes back from the fetching of ec2
data, instead of retrying immediatly stop the fetching process
and in the userdata fetching function handle this case as a
special case of no userdata being fetched (an empty string
in this case).
|
|
|
|
Couple of things here:
* do not re-try on user-data (404 means 'not here')
* re-generate headers on retry requests
LP: #1172742
|
|
the previous implementation of url_helper.readurl() would default
to allow_redirects being true.
So, for backwards compat, we should keep that behavior.
|
|
if the error is an ssl error, its extremely unlikely that it would be fixed by
waiting a few seconds and trying again.
|
|
* cloudinit/distros/parsers/resolv_conf.py
added some pylint overrides with 'plXXXXX' syntax.
example: # pl51222 pylint: disable=E0102
The pl51222 there means: http://www.logilab.org/ticket/51222
This specific issue is present in 12.04 pylint, but not 13.04.
* pylint doesn't like the requests special handling we have.
which makes sense as it is only checking versus one specific version.
* general pep8 and pylint cleanups.
|
|
in earlier versions of requests (pre 0.10.8).
|
|
|
|
|
|
|
|
|
|
1. Handle our own retries (so that we can sleep in between)
2. Cleanup the url so that partially invalid (no scheme) urls
will continue working.
3. Allow redirects option now a param.
|
|
|
|
since it provides the same support (and it uses
that library). Urllib3 doesn't seem to exist as
its own package (requests hides it) so solve that
by using requests and handling its oddness ourself.
|
|
userdata and metadata. Add in this crawling functionality
to the ec2_utils module that will fully crawl (not lazily)
the ec2 metadata and parse it in the same manner as boto.
1. Make the ec2 datasource + cloudstack now call into these.
2. Fix phone_home due to urllib3 change (TBD)
|
|
use ssl correctly (they can not do validation due to
the underlying usage of urllib/httplib). Adjust to
use urllib3 which can in cloud-init url fetching. For
now also take the metadata/userdata extraction code
from boto (small amount) and fix it in a new local file
called 'ec2_utils'.
|
|
In the event of a 403 (Unauthorized) in oauth, try set a 'oauth_clockskew'
variable. In future headers, use a time created by 'time.time() +
self.oauth_clockskew'. The idea here is that if the local time is bad (or even
if the server time is bad) we will essentially use something that should be
similar to the remote clock.
This fixes LP: #978127.
LP: #978127
|
|
cc_rightscale_userdata.py is likely to not be found, so its WARN
was annoying in that it got to console output.
The same is true of url_helper. Not all uses of readurl were
fatal or justified to warn if the content was not there.
|
|
2. Adjust comment on sources list from depends
3. For the /etc/timezone 'writing', add a header that says created by cloud-init
|
|
|
|
efficent
|
|
|
|
|
|
also post data...
|
|
a. This allows for more properties to be added as needed in the future, instead of being very restrictive.
2. Fix up all uses of the url reading to now use this new response object.
3. Also fixup user data including, such that if no response actual occurs the url content is not further processed.
|
|
|
|
|
|
from being valid codes.
2. Log how much we will sleep for
3. If not headers are provided, add a set which will have a cloud init user agent + version
|
|
|
