summaryrefslogtreecommitdiff
path: root/cloudinit/sources/DataSourceOpenNebula.py
AgeCommit message (Collapse)Author
2019-01-28opennebula: also exclude epochseconds from changed environment varsChad Smith
In addition to EPOCHREALTIME there is also an EPOCHSECONDS environment variable that OpenNebula needs to exclude as it is expected to change. This commit supplements the other exclusion in commit d1a2fe7307e9cf2251d1f9a666c12d71d3f522d6. Without this fix, unittests will intermittently fail if parse_shell_config is run across a timing boundary where the EPOCHSECONDS changes mid-test. LP: #1813641
2019-01-26opennebula: exclude EPOCHREALTIME as known bash env variable with a deltaChad Smith
This branch is needed to allow cloud-init to sbuild on Ubuntu Disco. OpenNebula:parse_shell_config tries to do a comparison of bash environment values, excluding expected environment variables which are known to change. Bash on Ubuntu Disco surfaces a new EPOCHREALTIME environment variable which wasn't in previous bash environments, this var needs to be ignored by parse_shell_config too. LP: #1813383
2018-10-09instance-data: Add standard keys platform and subplatform. Refactor ec2.Chad Smith
Add the following instance-data.json standardized keys: * v1._beta_keys: List any v1 keys in beta development, e.g. ['subplatform']. * v1.public_ssh_keys: List of any cloud-provided ssh keys for the instance. * v1.platform: String representing the cloud platform api supporting the datasource. For example: 'ec2' for aws, aliyun and brightbox cloud names. * v1.subplatform: String with more details about the source of the metadata consumed. For example, metadata uri, config drive device path or seed directory. To support the new platform and subplatform standardized instance-data, DataSource and its subclasses grew platform and subplatform attributes. The platform attribute defaults to the lowercase string datasource name at self.dsname. This method is overridden in NoCloud, Ec2 and ConfigDrive datasources. The subplatform attribute calls a _get_subplatform method which will return a string containing a simple slug for subplatform type such as metadata, seed-dir or config-drive followed by a detailed uri, device or directory path where the datasource consumed its configuration. As part of this work, DatasourceEC2 methods _get_data and _crawl_metadata have been refactored for a few reasons: - crawl_metadata is now a read-only operation, persisting no attributes on the datasource instance and returns a dictionary of consumed metadata. - crawl_metadata now closely represents the raw stucture of the ec2 metadata consumed, so that end-users can leverage public ec2 metadata documentation where possible. - crawl_metadata adds a '_metadata_api_version' key to the crawled ds.metadata to advertise what version of EC2's api was consumed by cloud-init. - _get_data now does all the processing of crawl_metadata and saves datasource instance attributes userdata_raw, metadata etc. Additional drive-bys: * unit test rework for test_altcloud and test_azure to simplify mocks and make use of existing util and test_helpers functions.
2018-08-03OpenNebula: Fix null gateway6Akihiko Ota
The OpenNebula data source generates an invalid netplan yaml file if the IPv6 gateway is not defined in context.sh. LP: #1768547
2018-05-29pyflakes: fix unused variable references identified by pyflakes 2.0.0.Scott Moser
A newer version of pyflakes (2.0.0) was released. It identifed some unused variables that version 1.6.0 did not identify. The change here merely fixes those unused variables.
2018-03-20datasources: fix DataSource subclass get_hostname method signatureChad Smith
DataSource.get_hostname call signature changed to allow for metadata_only parameter. The metadata_only=True parameter is passed to get_hostname during init-local stage in order to set the system hostname if present in metadata prior to initial network bring up. Fix subclasses of DataSource which have overridden get_hostname to allow for metadata_only param. LP: #1757176
2018-03-20OpenNebula: Update network to return v2 config rather than ENI.Akihiko Ota
OpenNebulaNetwork.gen_conf() was previously returning ENI format. This is updated to return netplan/v2 config. The changes here also adds support for IPv6 configuration distributed from OpenNebula and fixes some issues about nameserver information.
2018-03-02Change some list creation and population to literal.Rémy Léone
This will provide a small performance improvement and shorter code.
2018-01-24OpenNebula: Improve network configuration support.Akihiko Ota
Network configuration in OpenNebula would only work if the host correctly guessed the names of the devices in the guest. OpenNebula provided data in its context.sh like 'ETH0_NETWORK', but if the guest named devices differently then results were not predictable. This would occur with Predictable Network Interface Names. To address this, newer versions (of OpenNebula provide the mac address ETH0_MAC. This function is present in 4.14 and documented officially in 5.0 docs. This provides support for reading the mac addresses from the context.sh. It also fixes cases where context.sh provided a field (ETH0_NETWORK or ETH0_MASK) with a empty string. Previously the empty string would be used rather than falling back to the default. LP: #1719157, #1716397, #1736750
2017-12-15lint: Fix lints seen by pylint version 1.8.1.Chad Smith
This branch resolves lints seen by pylint revision 1.8.1 and updates our pinned tox pylint dependency used by our tox pylint target.
2017-12-05Datasources: Formalize DataSource get_data and related properties.Chad Smith
Each DataSource subclass must define its own get_data method. This branch formalizes our DataSource class to require that subclasses define an explicit dsname for sourcing cloud-config datasource configuration. Subclasses must also override the _get_data method or a NotImplementedError is raised. The branch also writes /run/cloud-init/instance-data.json. This file contains all meta-data, user-data and vendor-data and a standardized set of metadata keys in a json blob which other utilities with root-access could make use of. Because some meta-data or user-data is potentially sensitive the file is only readable by root. Generally most metadata content types should be json serializable. If specific keys or values are not serializable, those specific values will be base64encoded and the key path will be listed under the top-level key 'base64-encoded-keys' in instance-data.json. If json writing fails due to other TypeErrors or UnicodeDecodeErrors, a warning log will be emitted to /var/log/cloud-init.log and no instance-data.json will be created.
2017-04-21pylint: fix all logging warningsJoshua Powers
This will change all instances of LOG.warn to LOG.warning as warn is now a deprecated method. It will also make sure any logging uses lazy logging by passing string format arguments as function parameters.
2017-03-24test: add running of pylintJoshua Powers
Now tox will run pylint. The .pylintrc file sets pylint to only produce errors, and will ignore certain classes that are known problematic (six).
2016-12-22LICENSE: Allow dual licensing GPL-3 or Apache 2.0Jon Grimm
This has been a recurring ask and we had initially just made the change to the cloud-init 2.0 codebase. As the current thinking is we'll just continue to enhance the current codebase, its desirable to relicense to match what we'd intended as part of the 2.0 plan here. - put a brief description of license in LICENSE file - put full license versions in LICENSE-GPLv3 and LICENSE-Apache2.0 - simplify the per-file header to reference LICENSE - tox: ignore H102 (Apache License Header check) Add license header to files that ship. Reformat headers, make sure everything has vi: at end of file. Non-shipping files do not need the copyright header, but at the moment tests/ have it.
2016-10-19OpenNebula: replace 'ip' parsing with cloudinit.net usage.Scott Moser
Replace the parsing of 'ip' to get a link and mac address list in OpenNebula's datasource with usage of cloudinit.net. This makes test cases there not depend on 'ip' availability and also uses common code.
2016-08-23add install option for openrcMatthew Thode
Adds an install option for for OpenRC init scripts. I've also restricted installing tests more correctly. Also, don't hardcode the path to ip (/bin/ip on gentoo).
2016-06-15fix some errors reported by pylintScott Moser
pylint --errors-only found several errors. Some of the changes here represent real errors, others just code that pylint did not like.
2016-06-02re-add the 'Net' classes for datasourcesScott Moser
When the .pkl file is loaded, the module that it is loaded from must have the same symbol. Ie, if booted once and got DataSourceConfigDriveNet then upgraded and rebooted, then next boot would show Can't get attribute 'DataSourceConfigDriveNet'
2016-05-25commit to push for fear of loss.Scott Moser
== background == DataSource Mode (dsmode) is present in many datasources in cloud-init. dsmode was originally added to cloud-init to specify when this datasource should be 'realized'. cloud-init has 4 stages of boot. a.) cloud-init --local . network is guaranteed not present. b.) cloud-init (--network). network is guaranteed present. c.) cloud-config d.) cloud-init final 'init_modules' [1] are run "as early as possible". And as such, are executed in either 'a' or 'b' based on the datasource. However, executing them means that user-data has been fully consumed. User-data and vendor-data may have '#include http://...' which then rely on the network being present. boothooks are an example of the things run in init_modules. The 'dsmode' was a way for a user to indicate that init_modules should run at 'a' (dsmode=local) or 'b' (dsmode=net) directly. Things were further confused when a datasource could provide networking configuration. Then, we needed to apply the networking config at 'a' but if the user had provided boothooks that expected networking, then the init_modules would need to be executed at 'b'. The config drive datasource hacked its way through this and applies networking if *it* detects it is a new instance. == Suggested Change == The plan is to 1. incorporate 'dsmode' into DataSource superclass 2. make all existing datasources default to network 3. apply any networking configuration from a datasource on first boot only apply_networking will always rename network devices when it runs. for bug 1579130. 4. run init_modules at cloud-init (network) time frame unless datasource is 'local'. 5. Datasources can provide a 'first_boot' method that will be called when a new instance_id is found. This will allow the config drive's write_files to be applied once. Over all, this will very much simplify things. We'll no longer have 2 sources like DataSourceNoCloud and DataSourceNoCloudNet, but would just have one source with a dsmode. == Concerns == Some things have odd reliance on dsmode. For example, OpenNebula's get_hostname uses it to determine if it should do a lookup of an ip address. == Bugs to fix here == http://pad.lv/1577982 ConfigDrive: cloud-init fails to configure network from network_data.json http://pad.lv/1579130 need to support systemd.link renaming of devices in container http://pad.lv/1577844 Drop unnecessary blocking of all net udev rules
2016-03-04Apply pep8, pyflakes fixes for python2 and 3Scott Moser
Update make check target to run pep8 and run pyflakes or pyflakes3 depending on the value of 'PYVER'. This way the python3 build environment does not need python2 and vice versa. Also have make check run the 'yaml' test. tox: have tox run pep8 in the pyflakes
2016-03-03Update pep8 runner and fix pep8 issuesRyan Harper
2015-05-01fix 'Make pyflakes'Scott Moser
2015-01-27Respond to review:Barry Warsaw
- Refactor both the base64 encoding and decoding into utility functions. Also: - Mechanically fix some other broken untested code.
2015-01-27Respond to review:Barry Warsaw
- Just use util.load_file() instead of yet another way to open and read the file.
2015-01-26Another handling of b64decode.Barry Warsaw
Also, restore Python 2 compatibility.
2015-01-26* More str/bytes fixes.Barry Warsaw
* Temporarily skip the MAAS tests in py3 since they need to be ported to oauthlib.
2015-01-22Low hanging Python 3 fruit.Barry Warsaw
2014-08-26further remove evidence of pylint.Scott Moser
This just removes comments '# pylint:' things and other code remnents of pylint.
2014-03-27change 'default' encoding to be "None"Scott Moser
Instead of just trying to see if userdata decodes as the indication that it should be encoded, the user must explicitly set this. The "just try it" will fail in the case where the user had other use of user-data and wanted a blob of data to go through unrecognized by cloud-init. In cases where there can be mistake in automatic behavior, and some users may be relaying on old behavior, its best to just require explicit use.
2014-03-25Added base64 decoding of user data for OpenNebula.Enol Fernandez
2014-01-16DataSourceOpenNebula:parse_shell_config skip 'SECONDS' var if seenScott Moser
SECONDS is a special variable in bash, it gets set to the time the shell has been alive. This would cause us to fail randomly (if the process happened to take more than 1 second, then SECONDS would be defined).
2013-09-10some cleanups and changesScott Moser
* use util.subp from inside parse_shell_config, and adjust exception handling accordingly. * add 'switch_user_cmd' as a callback function to pass to parse_shell_config, which allows us to mock this to avoid 'sudo' when running test cases. Basically the test cases just return '[]' here. * fix some pylint * handle empty 'content' in parse_shell_config and remove the protection that was present.
2013-09-10Fix detection of ETHx_IP context variable, add test.Vlastimil Holer
2013-09-10Configurable OpenNebula::parseuser. Seed search dir+dev merge. ↵Vlastimil Holer
Eat shell parser error output. Few tests for tests for get_data.
2013-09-10Replace RE context.sh parser with Scott's rewrite of bash dumper. Upper case ↵Vlastimil Holer
context variable names.
2013-09-05PEP8 and Pylint fixes. Move context.sh "parser" into separateVlastimil Holer
function. Fix fetching user specified dsmode (from context). Rename context_sh->context. Reuse unittests.helpers.populate_dir.
2013-09-04Search for contextualization CDROM by LABEL=CONTEXTVlastimil Holer
2013-09-04Fix RE matching context variables. Test cleanups.Vlastimil Holer
2013-09-04Apply parse.diff by Javier Fontan <jfontan@opennebula.org>Vlastimil Holer
2013-04-02PEP8 fixes.Vlastimil Holer
2013-02-21Remove TODOVlastimil Holer
2013-02-20Minor OpenNebula DS cleanups (style, dsmode, static network).Vlastimil Holer
2013-02-19Context.sh parsing cleanups, fix single quotes handling in multiword variables.Vlastimil Holer
2013-02-19Name unification Context.*Device -> Context.*DiskVlastimil Holer
2013-02-19Change network ifaces detection from ifconfig to ip command.Vlastimil Holer
2013-01-04Append DNS related stuff to network interfaces configuration. Minor cleanups.Vlastimil Holer
2013-01-02Add OpenNebula.org copyright.Vlastimil Holer
2012-12-21Add OpenNebula contextualization options to cloud-initJavi Fontan
2012-12-20Fix exception handlers for data read. Fix name read_context_disk_dir.Vlastimil Holer
2012-12-20Change subp exception handling to util.ProcessExecutionErrorVlastimil Holer