summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2019-09-18docs: doc8 fixes for instancedata pageJoshua Powers
The huge table was scrolling off the page for me and it made more sense to break this up and have sections for each item anyway.
2019-09-18docs: clean up formatting, organize boot pageJoshua Powers
2019-09-17net: add is_master check for filtering device listRyan Harper
Some network devices are transformed into a bond via kernel magic and do not have the 'bonding' sysfs attribute, but like a bond they have a duplicate MAC of other bond members. On Azure Advanced Networking SRIOV devices are auto bonded and will have the same MAC as the HyperV nic. We can detect this via the 'master' sysfs attribute in the device sysfs path and this patch adds this to the list of devices we ignore when enumerating device lists. LP: #1844191
2019-09-14docs: more complete list of availabilityJoshua Powers
2019-09-14docs: start FAQ pageJoshua Powers
Takeover the moreinfo.rst page and update initially with pages a user can use to learn about cloud-init. For now, other docs and presentations about cloud-init. Goal is to use this page to collect very commonly asked questions.
2019-09-14docs: cleanup output & order of datasource pageJoshua Powers
2019-09-11Brightbox: restrict detection to require full domain match .brightbox.comScott Moser
The detection for brightbox in both ds-identify and in identify_brightbox would incorrectly match the domain 'bobrightbox', which is not a brightbox platform. The fix here is to restrict matching to '*.brightbox.com' rather than '*brightbox.com' Also, while here remove a url to bug 1661693 which added the knowledge of brightbox.
2019-09-11VMWware: add option into VMTools config to enable/disable custom script.Xiaofeng Wang
VMWware customization already has support to run a custom script during the VM customization. Adding this option allows a VM administrator to disable the execution of customization scripts. If set the script will not execute and the customization status is set to GUESTCUST_ERROR_SCRIPT_DISABLED.
2019-09-09net,Oracle: Add support for netfailover detectionRyan Harper
Add support for detecting netfailover[1] device 3-tuple in networking layer. In the Oracle datasource ensure that if a provided network config, either fallback or provided config includes a netfailover master to remove any MAC address value as this can break under 3-netdev as the other two devices have the same MAC. 1. https://www.kernel.org/doc/html/latest/networking/net_failover.html
2019-09-09atomic_helper: add DEBUG logging to write_fileDaniel Watkins
LP: #1843276
2019-09-07doc: document doc, create makefile and tox targetJoshua Powers
* Create makefile and tox targets for documentation building and testing to better replicate the live web docs using the same theme. * Created docs.rst to explain how to build and contribute to documentation with style guide and tips. * doc/rtd/conf.py:     * Add copyright to rtd config     * Use Sphinx's RTD theme to replicate actual docs
2019-09-06.gitignore: ignore files produced by package buildsDaniel Watkins
2019-09-04docs: fix whitespace, spelling, and line lengthJoshua Powers
2019-09-03docs: remove unnecessary file in doc directoryJoshua Powers
2019-08-28Oracle: Render secondary vnic IP and MTU values onlyRyan Harper
When rendering secondary vnic configuration from IMDS, only emit configuration for the IP and MTU values only. Add support to mutate either a v1 or a v2 network_config input.
2019-08-28exoscale: fix sysconfig cloud_config_modules overridesChad Smith
Make sure Exoscale supplements or overrides existing system config setting cloud_config_modules instead of replacing it with a one item list set-passords LP: #1841454
2019-08-22net/cmdline: refactor to allow multiple initramfs network config sourcesDaniel Watkins
This refactors read_initramfs_config to support multiple different types of initramfs network configuration. It introduces an InitramfsNetworkConfigSource abstract base class. There is currently a single sub-class, KlibcNetworkConfigSource, which contains the logic which previously was directly within read_initramfs_config.
2019-08-22ubuntu-drivers: call db_x_loadtemplatefile to accept NVIDIA EULAChad Smith
Emit a script allowing cloud-init to set linux/nvidia/latelink debconf selection to true. This avoids having to call debconf-set-selections and allows cloud-init to pre-confgure linux-restricted-modules to link NVIDIA drivers to the running kernel. Cloud-init loads this debconf template and sets the value to true in the debconf database by sourcing debconf's /usr/share/debconf/confmodule and uses db_x_loadtemplatefile to register cloud-init's setting for linux/nvidia/latelink. LP: #1840080
2019-08-20Add missing #cloud-config comment on first example in documentation.Florian Müller
Since this is the first code snippet some users might see, they could end up in the same situation like me today when they wonder why their yaml user config is not working at all.
2019-08-19ubuntu-drivers: emit latelink=true debconf to accept nvidia eulaChad Smith
To accept NVIDIA EULA, cloud-init needs to emit latelink=true debconf setting to the linux-restricted-modules package to allow NVIDIA drivers to properly link to the running kernel. LP: #1840080
2019-08-19DataSourceOracle: prefer DS network config over initramfsDaniel Watkins
The Oracle platform provides networking configuration from two sources: * the primary interface configuration comes from the initramfs, because Oracle instance all iSCSI boot * secondary interface configuration comes from an IMDS accessed over HTTP As we need to combine these two sources of network configuration, the default "prefer initramfs config over data source config" behaviour isn't appropriate; we would never get the IMDS interfaces via that route. Instead, the Oracle data source has code to combine these two sources, so we prefer its network configuration over the initramfs configuration. (This is not appropriate default behaviour, because _in general_ data sources won't know how to merge initramfs-provided configuration into their provided configuration, so switching this order for all data sources would result in initramfs configuration being discarded on any data source that implements network_config.)
2019-08-19format.rst: add text/jinja2 to list of content types (+ cleanups)Daniel Watkins
The cleanups, specifically, are to sort the list of content types, and remove trailing whitespace.
2019-08-19Add GitHub pull request template to point people at hacking docDaniel Watkins
This will hopefully avoid people submitting PRs against our mirror repo. (Thanks to GitHub user @max06 for this suggestion!)
2019-08-16cloudinit/distros/parsers/sys_conf: add docstring to SysConfDaniel Watkins
2019-08-15pyflakes: remove unused variableJoshua Powers
2019-08-14Azure: Record boot timestamps, system information, and diagnostic eventsAnh Vo
Collect and record the following information through KVP:  + timestamps related to kernel initialization and systemd activation    of cloud-init services  + system information including cloud-init version, kernel version,    distro version, and python version  + diagnostic events for the most common provisioning error issues    such as empty dhcp lease, corrupted ovf-env.xml, etc. + increasing the log frequency of polling IMDS during reprovision.
2019-08-14DataSourceOracle: configure secondary NICs on Virtual MachinesDaniel Watkins
Oracle Cloud Infrastructure's Instance Metadata Service provides network configuration information for non-primary NICs. This commit introduces support, on Virtual Machines[0], for fetching that network metadata, converting it to v1 network-config[1] and combining it into the network configuration generated for the primary interface. By default, this behaviour is not enabled. Configuring the Oracle datasource to `configure_secondary_nics` enables it: datasource: Oracle: configure_secondary_nics: true Failures to fetch and generate secondary NIC configuration will log a warning, but otherwise will not affect boot. [0] The expected use of the IMDS-provided network configuration is substantially different on Bare Metal Machines, so support for that will be addressed separately. [1] This is v1 config, because cloudinit.net.cmdline generates v1 config and we need to integrate the secondary NICs into that configuration.
2019-08-14distros: fix confusing variable namesDaniel Watkins
Building the subp arguments for a `useradd` call in a variable named `adduser_cmd` is extremely confusing; let's not do that. (This also changes the snap and freebsd variables to something more apropos.)
2019-08-13azure/net: generate_fallback_nic emits network v2 config instead of v1Chad Smith
The function generate_fallback_config is used by Azure by default when not consuming IMDS configuration data. This function is also used by any datasource which does not implement it's own network config. This simple fallback configuration sets up dhcp on the most likely NIC. It will now emit network v2 instead of network v1. This is a step toward moving all components talking in v2 and allows us to avoid costly conversions between v1 and v2 for newer distributions which rely on netplan.
2019-08-09Add support for publishing host keys to GCE guest attributesRick Wright
This adds an empty publish_host_keys() method to the default datasource that is called by cc_ssh.py. This feature can be controlled by the 'ssh_publish_hostkeys' config option. It is enabled by default but can be disabled by setting 'enabled' to false. Also, a blacklist of key types is supported. In addition, this change implements ssh_publish_hostkeys() for the GCE datasource, attempting to write the hostkeys to the instance's guest attributes. Using these hostkeys for ssh connections is currently supported by the alpha version of Google's 'gcloud' command-line tool. (On Google Compute Engine, this feature will be enabled by setting the 'enable-guest-attributes' metadata key to 'true' for the project/instance that you would like to use this feature for. When connecting to the instance for the first time using 'gcloud compute ssh' the hostkeys will be read from the guest attributes for the instance and written to the user's local known_hosts file for Google Compute Engine instances.)
2019-08-08New data source for the Exoscale.com cloud platformChris Glass
- dsidentify switches to the new Exoscale datasource on matching DMI name - New Exoscale datasource added Signed-off-by: Mathieu Corbin <mathieu.corbin@exoscale.ch>
2019-08-08doc: remove intersphinx extensionDaniel Watkins
We don't use it in our doc builds, and if sphinx.pocoo.org is down then it causes doc build issues.
2019-08-07cc_set_passwords: rewrite documentationDaniel Watkins
What we had previously was inaccurate in a few respects. LP: #1838794
2019-07-26net/cmdline: split interfaces_by_mac and init network config determinationDaniel Watkins
Previously "cmdline" network configuration could be either user-specified network-config=... configuration data, or initramfs-provided configuration data. Before data sources could modify the order in which network config sources were considered, this conflation didn't matter (and, indeed, in the default data source configuration it will continue to not matter). However, it _is_ desirable for a data source to be able to specify that its network configuration should be preferred over the initramfs-provided network configuration but still allow explicit network-config=... configuration passed to the kernel cmdline to continue to override both of those sources. (This also modifies the Oracle data source to use read_initramfs_config directly, which is effectively what it was using read_kernel_cmdline_config for previously.)
2019-07-23stages: allow data sources to override network config source orderDaniel Watkins
Currently, if a platform provides any network configuration via the "cmdline" method (i.e. network-data=... on the kernel command line, ip=... on the kernel command line, or iBFT config via /run/net-*.conf), the value of the data source's network_config property is completely ignored. This means that on platforms that use iSCSI boot (such as Oracle Compute Infrastructure), there is no way for the data source to configure any network interfaces other than those that have already been configured by the initramfs. This change allows data sources to specify the order in which network configuration sources are considered. Data sources that opt to use this mechanism will be expected to consume the command line network data and integrate it themselves. (The generic merging of network configuration sources was considered, but we concluded that the single use case we have presently (a) didn't warrant the increased complexity, and (b) didn't give us a broad enough view to be sure that our generic implementation would be sufficiently generic. This change in no way precludes a merging strategy in future.)
2019-07-18cloud_tests: updates and fixesRyan Harper
- Update paramiko and cryptography module versions (2.4.2) to address issues with algo and deprecation warnings. - Modify ssh keypair generation to work with updated paramiko - tools/xkvm sync with newer version from curtin - Update NoCloudKvm instance.py to work with updated xkvm - pass -name to instance, useful for debugging on shared host - Add cache_mode platform config; default to cache=none,aio=native - Switch to yaml.safe_load() in platforms.py
2019-07-18Fix bug rendering MTU on bond or vlan when input was netplan.Scott Moser
If input to network_state.parse_net_config_data was netplan (v2 yaml) then the network state would lose the mtu information on bond or vlan. LP: #1836949
2019-07-17net: update net sequence, include wait on netdevs, opensuse netrules pathRyan Harper
On systems with many interfaces, processing udev events may take a while. Cloud-init expects devices included in a provided network-configuration to be present when attempting to configure them. This patch adds a step in net configuration where it will check for devices provided in the configuration and if not found, issue udevadm settle commands to wait for them to appear. Additionally, the default path for udev persistent network rules 70-persistent-net.rules may also be written to systems which include the 75-net-generator.rules. During boot, cloud-init and the generator may race and interleave values causing issues. OpenSUSE will now use a newer file, 85-persistent-net-cloud-init.rules which will take precedence over values created by 75-net-generator and avoid collisions on the same file. LP: #1817368
2019-07-17Release 19.2Ryan Harper
Bump the version in cloudinit/version.py to be 19.2 and update ChangeLog. LP: #1836921
2019-07-16net: add rfc3442 (classless static routes) to EphemeralDHCPRyan Harper
The EphemeralDHCP context manager did not parse or handle rfc3442 classless static routes which prevented reading datasource metadata in some clouds. This branch adds support for extracting the field from the leases output, parsing the format and then adding the required iproute2 ip commands to apply (and teardown) the static routes. LP: #1821102
2019-07-16templates/ntp.conf.debian.tmpl: fix missing newline for poolsRyan Harper
The debian ntp.conf template did not contain a newline for the comment used to mark the rendered ntp pools configured. This resulted in an invalid line: '# poolspool 0.int.pool.ntp.org iburst' rather than: '# pools  pool 0.int.pool.ntp.org iburst' This patch fixes the template and updates the unittest to verify that the rendered templates puts servers and pools at the beginning of a line. LP: #1836598
2019-07-16Support netplan renderer in Arch LinuxConrad Hoffmann
Support is for now implemented in such a way that it will fall back to the old `_write_network()` if netplan is not available on the image.
2019-07-16Fix typo in publicly viewable documentation.David Medberry
2019-07-16Add a cdrom size checker for OVF ds to ds-identifyPengpeng Sun
With a large size ISO file attached to iso dev, ds-identify might grep it entirely if iso dev is ISO9660, it takes very long time to start OS. Resolve this by: - Adding a checker to read the ISO size (from sysfs). If the size of the ISO filesystem is > 10MiB then the ISO will be ignored (logged as oversized). - Move the ovf vmware guest customization checker to be ahead of cdrom ovf checker, so no need check the ISO size if vmware guest customization is enabled. LP: #1806701
2019-07-16VMWare: Trigger the post customization script via cc_scripts module.Xiaofeng Wang
cloud-init does not trigger reboots of a VM therefore adding custom scripts to rc.local does not execute the post scripts. This patch moves post-scripts into per-instance scripts dir and has cc_scripts module run the post-scripts. Also in this branch: - Remove the sh interpreter and execute the customization script directly. - Update the unit test. LP: #1833192
2019-07-15Cloud-init analyze module: Added ability to analyze boot events.Sam Gilson
This branch introduces a new command line feature for cloud-init. Currently, the cloud-init module has the capability to analyze events in cloud-init.log in three ways: 'show', 'blame', 'dump'. These changes add a fourth capability, called 'boot'. Running the command 'cloud-init analyze boot' will provide the user three timestamps. 1) Timestamp for when the kernel starts initializing. 2) Timestamp for when the kernel finishes its initialization. 3) Timestamp for when systemd activates cloud-init. This feature enables cloud-init users to analyze different boot phases. This would aid in debugging performance issues related to cloud-init startup or tracking regression.
2019-07-15Update debian eni network configuration location, retain Ubuntu settingJanos Lenart
On Debian, ifupdown uses `source-directory /etc/network/interfaces.d` (for new installs) to include files. https://salsa.debian.org/debian/ifupdown/blob/master/debian/postinst#L23 The current filename, 50-cloud-init.cfg, does not match against the RE that is used to scan the directory for configurations (ASCII upper- and lower-case letters, ASCII digits, ASCII underscores, and ASCII minus-hyphens): https://salsa.debian.org/debian/ifupdown/blob/master/interfaces.5.pre#L122 Of course many installations use `source /etc/network/interfaces.d/*`, but not all.
2019-07-15net: skip bond interfaces in get_interfacesStanislav Makar
bonds may inherit mac address from a physical interface LP: #1812857
2019-07-10Fix a couple of issues raised by a coverity scanDaniel Watkins
* cc_lxd: fix copy/paste error in debug logging * DataSourceCloudSigma: remove unreachable code * This unreachable code was introduced in a refactor (in 2015) which removed the need for an exception handler, but retained the logging from the exception handler as an unreachable fall-through.
2019-07-03Add missing dsname for Hetzner Cloud datasourceMarkus Schade