summaryrefslogtreecommitdiff
path: root/cloudinit/url_helper.py
AgeCommit message (Collapse)Author
2014-02-13wait less for the metadata service (by default)Scott Moser
Waiting around for a metadata service in a given datasource means that if its not there all the subsequent datasources have to wait, and boot is slowed down. As it is right now, EC2 is the only one that has the right to wait. In the past, we had to wait around for the EC2 metadata service. I really do not want to extend that courtesy to other cloud platforms. A network based metadata service should be up as soon as networking is up.
2014-02-08Spacing and comment cleanupJoshua Harlow
2014-02-08Update requests ssl not supported messageJoshua Harlow
2014-02-08Remove HEAD usage and other small adjustmentsJoshua Harlow
2014-02-07Remerged with trunkJoshua Harlow
2014-02-01Add a openstack specific datasourceJoshua Harlow
Openstack has a unique derivative datasource that is gaining usage. Previously the config drive datasource provided part of this functionality as well as the ec2 datasource, but since new functionality is being added to openstack is seems benefical to combine the used parts into one datasource just made for handling openstack deployments. This patch factors out the common logic shared between the config drive and the openstack metadata datasource and places that in a shared helper file and then creates a new openstack datasource that readers from the openstack metadata service and refactors the config drive datasource to use this common logic.
2014-01-24read_file_or_url: raise UrlError with 404 on ENOENTScott Moser
This makes it easier to call read_file_or_url and handle file or url errors. Now read_file_or_url will raise a UrlError in either case on errors.
2014-01-23Use the right exceptionJoshua Harlow
2014-01-23Skip retry and continued fetch of userdata when NOT_FOUNDJoshua Harlow
When a 404 http code comes back from the fetching of ec2 data, instead of retrying immediatly stop the fetching process and in the userdata fetching function handle this case as a special case of no userdata being fetched (an empty string in this case).
2013-05-03Fix a feature that doesn't yet exist on python 2.6Joshua Harlow
2013-04-25re-work maas datasource and headers call backsScott Moser
Couple of things here: * do not re-try on user-data (404 means 'not here') * re-generate headers on retry requests LP: #1172742
2013-03-19set 'allow_redirects' to True by defaultScott Moser
the previous implementation of url_helper.readurl() would default to allow_redirects being true. So, for backwards compat, we should keep that behavior.
2013-03-19do not bother retrying on ssl errorsScott Moser
if the error is an ssl error, its extremely unlikely that it would be fixed by waiting a few seconds and trying again.
2013-03-19appease pylint and pep8Scott Moser
* cloudinit/distros/parsers/resolv_conf.py added some pylint overrides with 'plXXXXX' syntax. example: # pl51222 pylint: disable=E0102 The pl51222 there means: http://www.logilab.org/ticket/51222 This specific issue is present in 12.04 pylint, but not 13.04. * pylint doesn't like the requests special handling we have. which makes sense as it is only checking versus one specific version. * general pep8 and pylint cleanups.
2013-03-13Fix how the http error doesn't always have the response attachedJoshua Harlow
in earlier versions of requests (pre 0.10.8).
2013-03-13Update to handle requests >= 1.0 which doesn't use the config dict.Joshua Harlow
2013-02-23Fix the maas callback mechanism now that requests is used.harlowja
2013-02-23Get tests working and further adjustments.harlowja
2013-02-19Continue working on integrating requests.harlowja
2012-10-22More cleanups for using the requests module.Joshua Harlow
1. Handle our own retries (so that we can sleep in between) 2. Cleanup the url so that partially invalid (no scheme) urls will continue working. 3. Allow redirects option now a param.
2012-10-21Fix the checking and capturing of exceptions.Joshua Harlow
2012-10-21Use the python requests lib instead of urllib3Joshua Harlow
since it provides the same support (and it uses that library). Urllib3 doesn't seem to exist as its own package (requests hides it) so solve that by using requests and handling its oddness ourself.
2012-10-19Remove the need for boto just for fetching the Joshua Harlow
userdata and metadata. Add in this crawling functionality to the ec2_utils module that will fully crawl (not lazily) the ec2 metadata and parse it in the same manner as boto. 1. Make the ec2 datasource + cloudstack now call into these. 2. Fix phone_home due to urllib3 change (TBD)
2012-10-18Start fixing how boto and our url fetching can notJoshua Harlow
use ssl correctly (they can not do validation due to the underlying usage of urllib/httplib). Adjust to use urllib3 which can in cloud-init url fetching. For now also take the metadata/userdata extraction code from boto (small amount) and fix it in a new local file called 'ec2_utils'.
2012-09-24DataSourceMAAS: if a oauth request fails due to 403 try updating local timeScott Moser
In the event of a 403 (Unauthorized) in oauth, try set a 'oauth_clockskew' variable. In future headers, use a time created by 'time.time() + self.oauth_clockskew'. The idea here is that if the local time is bad (or even if the server time is bad) we will essentially use something that should be similar to the remote clock. This fixes LP: #978127. LP: #978127
2012-07-09url_helper.py, cc_rightscale_userdata.py: reduce WARN to DEBUGScott Moser
cc_rightscale_userdata.py is likely to not be found, so its WARN was annoying in that it got to console output. The same is true of url_helper. Not all uses of readurl were fatal or justified to warn if the content was not there.
2012-06-211. Move all info() logging methods to debug() harlowja
2. Adjust comment on sources list from depends 3. For the /etc/timezone 'writing', add a header that says created by cloud-init
2012-06-20Massive pylint + pep8 fixups!Joshua Harlow
2012-06-20Use comparisons instead of xrange, until python 3 when it should be more ↵harlowja
efficent
2012-06-19Cleanup pylint warning about line length.Joshua Harlow
2012-06-18Show the timeout (useful to see this when debugging)Joshua Harlow
2012-06-18Adjust log message to say "open" instead of 'read' since this function can ↵Joshua Harlow
also post data...
2012-06-171. Add a url response class that urlreading now returns (instead of a tuple). harlowja
a. This allows for more properties to be added as needed in the future, instead of being very restrictive. 2. Fix up all uses of the url reading to now use this new response object. 3. Also fixup user data including, such that if no response actual occurs the url content is not further processed.
2012-06-16Allow url fetching to catch any other type of exceptions that occurJoshua Harlow
2012-06-16Capture the exceptions and just re-throw the last one on errorsJoshua Harlow
2012-06-151. Modify the check http code function to allow or not allow redirect status ↵Joshua Harlow
from being valid codes. 2. Log how much we will sleep for 3. If not headers are provided, add a set which will have a cloud init user agent + version
2012-06-11Logging fixups + pylint fixups.Joshua Harlow
2012-06-09Add copyrightJoshua Harlow
2012-06-08Cleaning this up, logic should be more sound around attempts and retries now.Joshua Harlow
2012-06-07Add a file that just deals with url like functionsJoshua Harlow