| Age | Commit message (Collapse) | Author |
|---|
|
Mark as supported for reading some newer versions of openstack metadata:
2016-06-30 : Newton one
2016-10-06 : Newton two
2017-02-22 : Ocata
2018-08-27 : Rocky
|
|
Cloud-init was reading a list of versions from the OpenStack metadata
service (http://169.254.169.254/openstack/) and attempt to select the
newest known supported version. The problem was that the list
of versions was not being decoded, so we were comparing a list of
bytes (found versions) to a list of strings (known versions).
LP: #1792157
|
|
This adds a Oracle specific datasource that functions with OCI.
It is a simplified version of the OpenStack metadata server
with support for vendor-data.
It does not support the OCI-C (classic) platform.
Also here is a move of BrokenMetadata to common 'sources'
as this was the third occurrence of that class.
|
|
The OpenStack datasource in 18.3 changed to detect data in the
init-local stage instead of init-network and attempted to redetect
OpenStackLocal datasource on Oracle across reboots. The function
detect_openstack was added to quickly detect whether a platform is
OpenStack based on dmi product_name or chassis_asset_tag and it was
a bit too strict for Oracle in checking for 'OpenStack Nova'/'Compute'
DMI product_name.
Oracle's DMI product_name reports 'SAtandard PC (i440FX + PIIX, 1996)'
and DMI chassis_asset_tag is 'OracleCloud.com'.
detect_openstack function now adds 'OracleCloud.com' as a supported value
'OracleCloud.com' to valid chassis-asset-tags for the OpenStack
datasource.
LP: #1784685
|
|
OpenStack datasource is now discovered in init-local stage. In order to
probe whether OpenStack metadata is present, it performs a costly
sandboxed dhclient setup and metadata probe against http://169.254.169.254
for openstack data.
Cloud-init properly detects non-OpenStack on EC2, but it spends precious
time probing the metadata service also resulting in a confusing WARNING
log about 'metadata not present'. To avoid the wasted cycles, and
confusing warning, get_data will call a detect_openstack function to
quickly determine whether the platform looks like OpenStack before trying
to setup network to probe and crawl the metadata service.
LP: #1776701
|
|
Network has not yet been configured in the init-local stage so the
openstack datasource will use dhcp-client to temporarily obtain an ipv4
address and query the metadata service at http://169.254.169.254 to get
network_data.json configuration. If present, the datasource will return
network_config version 1 config based on that network_data.json content.
Previously OpenStack datasource only setup dhcp on the fallback interface
so this represents a change in behavior to react to the full config
provided by openstack.
Also significant to OpenStack is the separation of a _crawl_data operation
from get_data(). crawl_data walks the available metadata services and
returns a dict of discovered content. get_data consumes the crawled_data,
caches it in the datasource and reacts to that data.
/run/cloud-init/instance-data.json now published network_data.json or
ec2_metadata key if that data is present on any datasource.
The main reasons for the separation of crawl from get_data:
* Enable performance metrics of cloud-init's metadata crawls on each
* Enable cloud-init modules and scripts to query and consume metadata
content which may have updated/changed after cloud-init's initial cache
during instance boot. (Think hotplug)
Also generalize common logic to base DataSource class/module:
* Move to a common UNSET variable up into base datasource module fix EC2,
ConfigDrive, OpenStack, SmartOS to use the global.
* Drop get_url_settings from Ec2, CloudStack and OpenStack and generalize
DataSource.get_url_params(). Allow subclasses to override url_max_wait,
url_timeout and url_retries params.
* Rename get_network_metadata bool to perform_dhcp_setup as it designates
whether EphemeralDHCPv4 setup is required before crawling metadata.
LP: #1749717
|
|
On OpenSuSE 42.3, we would get errors running
tests/unittests/test_handler/test_handler_chef.py
- test_myhttps_nonet raises a UnmockedError
No mocking was registered, and real connections are not allowed
- test_myhttps_net raises SSLError
("bad handshake: SysCallError(32, 'EPIPE')",)
This fixes the errors by just using http instead of https.
Also it modifies the HttprettyTestCase to do the httpretty activate
and deactivate itself in setUp and tearDown. Then we don't have to
decorate individual test_ methods. Also, we set
httpretty.HTTPretty.allow_net_connect = False
Test cases here should not reach out to a network resource.
LP: #1771659
|
|
Each DataSource subclass must define its own get_data method. This branch
formalizes our DataSource class to require that subclasses define an
explicit dsname for sourcing cloud-config datasource configuration.
Subclasses must also override the _get_data method or a
NotImplementedError is raised.
The branch also writes /run/cloud-init/instance-data.json. This file
contains all meta-data, user-data and vendor-data and a standardized set
of metadata keys in a json blob which other utilities with root-access
could make use of. Because some meta-data or user-data is potentially
sensitive the file is only readable by root.
Generally most metadata content types should be json serializable. If
specific keys or values are not serializable, those specific values will
be base64encoded and the key path will be listed under the top-level key
'base64-encoded-keys' in instance-data.json. If json writing fails due to
other TypeErrors or UnicodeDecodeErrors, a warning log will be emitted to
/var/log/cloud-init.log and no instance-data.json will be created.
|
|
This missed mock in test_openstack resulted in a costly unit test timeout.
LP: #1714376
|
|
This moves the base test case classes into into cloudinit/tests and
updates all the corresponding imports.
|
|
This just moves flake8 and related tools up to newer versions and fixes
the complaints associated with that.
We added to the list of flake8 ignores:
H102: do not put vim info in source files
H304: no relative imports
Also updates and pins the following in the flake8 environment:
pep8: 1.7.0 => drop (although hacking still pulls it in).
pyflakes 1.1.0 => 1.5.0
hacking 0.10.2 => 0.13.0
flake8 2.5.4 => 3.3.0
pycodestyle none => 2.3.1
|
|
No shipping ubuntu has a python 3.4 that is less than 3.4.2.
Remove this workaround to avoid unnecessary complexity.
This reverts 04a60cf949.
|
|
This modifies get_data in DataSourceOpenStack.py to get the timeout
and retries values from the data source configuration, rather than
from keyword arguments. This permits get_data to use the same timeout
as other methods, and allows an operator to increase the timeout in
environments where the metadata service takes longer than five seconds
to respond.
LP: #1657130
Resolves: rhbz#1408589
|
|
This has been a recurring ask and we had initially just made the change to
the cloud-init 2.0 codebase. As the current thinking is we'll just
continue to enhance the current codebase, its desirable to relicense to
match what we'd intended as part of the 2.0 plan here.
- put a brief description of license in LICENSE file
- put full license versions in LICENSE-GPLv3 and LICENSE-Apache2.0
- simplify the per-file header to reference LICENSE
- tox: ignore H102 (Apache License Header check)
Add license header to files that ship.
Reformat headers, make sure everything has vi: at end of file.
Non-shipping files do not need the copyright header,
but at the moment tests/ have it.
|
|
An obvious fix for an issue raised by pyflakes 1.3.
|
|
Add vendor-data support to maas which will behave like the openstack
vendor-data does. Data returned from maas must be yaml loadable.
Also update the main in DataSourceMAAS to "just work" on a maas
deployed system.
LP: #1612313
|
|
Timeouts and retries were triggering so make it so
that tests do not use the typical timesouts and retries
so that the tests finish faster.
|
|
|
|
HTTPretty can causes hangs on Python 3.4.2 (and maybe Python 3.4.1), due
to a Python bug (fixed in Python 3.4.3). This works around the problem
in the appropriate Python versions.
See https://github.com/gabrielfalcao/HTTPretty/pull/193 and
https://github.com/gabrielfalcao/HTTPretty/issues/221 for details.
|
|
UrlResponse: biggest change... make readurl return bytes, making user
know what to do with it.
util: add load_tfile_or_url for loading text file or url
as read_file_or_url now returns bytes
ec2_utils: all meta-data is text, remove non-obvious string translations
DigitalOcean: adjust for ec2_utils
DataSourceGCE, DataSourceMAAS: user-data is binary other fields are text.
openstack.py: read paths without decoding to text. This is ok as paths
other than user-data are json, and load_json will handle
load_file still returns text, and that is what most things use.
|
|
to be behind trunk.
`tox -e py27` passes full test suite. Now to work on replacing mocker.
|
|
This adds a DataSource for DigitalOcean's metadata service. The service is
documented at https://developers.digitalocean.com/metadata/ .
|
|
There are a couple new test cases that are now erroring out.
- A usage of the non helper unit testcase base which is trigging
and error on python 2.6 due to lack of method, fix this by using
the base helper class.
- A freebsd distro test check thats looking for /etc/resolv.conf and
examining its contents, which won't exist due to our mocking routine
that does not allow that file to be read.
- A freebsd distro test where the distro class tries to call into
['ifconfig', '-a'] for a set of values, those values don't exist on
the machine I am running on (and likely others machines) so we should
mock the subp function out (that the distro class calls) and correctly
return values that will work for the testcase.
|
|
For now, this vendor data handling is just added to openstack.
However, in an effort to allow sanely handling of multi-part vendor-data
that is namespaced, we add openstack.convert_vendordata_json .
That basically takes whatever was loaded from vendordata and takes
the 'cloud-init' key if it is a dict. This way the author can
namespace cloud-init, basically telling it to ignore everything else.
|
|
instead of taking a version that they should look for,
the readers now just select the highest supported version.
definitely a use case later for having version= but nothing
is using it now.
|
|
|
|
|
|
This change works around a bug in CloudStack's EC2 api implementation.
That is filed upstream at [1].
The fix is safe for openstack and EC2 use cases as well.
EC2 and OpenStacks' EC2 metadata service both return a list with
access to either of:
<url_base>/latest/meta-data
<url_base>/latest/meta-data/
Additionally, the responses explicitly contain a trailing '/' for
items that have a child. The ec2_utils code then just re-uses the trailng
/ there. Thus, only the top level request for 'meta-data/' needs
the explicit fix.
This also changes test cases. Those test cases failed without fixing
them. If ever this regressed, those would fail again.
--
[1] https://issues.apache.org/jira/browse/CLOUDSTACK-7405
LP: #1356855
|
|
|
|
previously this would fail:
http_proxy=http://foo.bar make test
now it will pass. This works around a bug where httpretty is not able to
patch http operations if http_proxy is set.
https://github.com/gabrielfalcao/HTTPretty/issues/122
|
|
|
|
|
|
|
|
|
|
|
