Cloud-init: T2117: Updated to 20.1

  - Merge 20.1 version from the Canonical repository
  - Removed unneeded changes in datasources (now only OVF datasource is not equal to upstream's version)
  - Adapted cc_vyos module to new Cloud-init version
  - Changed Jenkinsfile to use build scripts, provided by upstream

1 files changed, 401 insertions, 0 deletions

diff --git a/ChangeLog b/ChangeLog
index 8fa6fdd4..dd4f7add 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,404 @@
+20.1
+ - ec2: Do not log IMDSv2 token values, instead use REDACTED (#219)
+   (LP: #1863943)
+ - utils: use SystemRandom when generating random password. (#204)
+   [Dimitri John Ledkov]
+ - docs: mount_default_files is a list of 6 items, not 7 (#212)
+ - azurecloud: fix issues with instances not starting (#205) (LP: #1861921)
+ - unittest: fix stderr leak in cc_set_password random unittest
+   output. (#208)
+ - cc_disk_setup: add swap filesystem force flag (#207)
+ - import sysvinit patches from freebsd-ports tree (#161) [Igor Galić]
+ - docs: fix typo (#195) [Edwin Kofler]
+ - sysconfig: distro-specific config rendering for BOOTPROTO option (#162)
+   [Robert Schweikert] (LP: #1800854)
+ - cloudinit: replace "from six import X" imports (except in util.py) (#183)
+ - run-container: use 'test -n' instead of 'test ! -z' (#202)
+   [Paride Legovini]
+ - net/cmdline: correctly handle static ip= config (#201)
+   [Dimitri John Ledkov] (LP: #1861412)
+ - Replace mock library with unittest.mock (#186)
+ - HACKING.rst: update CLA link (#199)
+ - Scaleway: Fix DatasourceScaleway to avoid backtrace (#128)
+   [Louis Bouchard]
+ - cloudinit/cmd/devel/net_convert.py: add missing space (#191)
+ - tools/run-container: drop support for python2 (#192) [Paride Legovini]
+ - Print ssh key fingerprints using sha256 hash (#188) (LP: #1860789)
+ - Make the RPM build use Python 3 (#190) [Paride Legovini]
+ - cc_set_password: increase random pwlength from 9 to 20 (#189)
+   (LP: #1860795)
+ - .travis.yml: use correct Python version for xenial tests (#185)
+ - cloudinit: remove ImportError handling for mock imports (#182)
+ - Do not use fallocate in swap file creation on xfs. (#70)
+   [Eduardo Otubo] (LP: #1781781)
+ - .readthedocs.yaml: install cloud-init when building docs (#181)
+   (LP: #1860450)
+ - Introduce an RTD config file, and pin the Sphinx version to the RTD
+   default (#180)
+ - Drop most of the remaining use of six (#179)
+ - Start removing dependency on six (#178)
+ - Add Rootbox & HyperOne to list of cloud in README (#176) [Adam Dobrawy]
+ - docs: add proposed SRU testing procedure (#167)
+ - util: rename get_architecture to get_dpkg_architecture (#173)
+ - Ensure util.get_architecture() runs only once (#172)
+ - Only use gpart if it is the BSD gpart (#131) [Conrad Hoffmann]
+ - freebsd: remove superflu exception mapping (#166) [Gonéri Le Bouder]
+ - ssh_auth_key_fingerprints_disable test: fix capitalization (#165)
+   [Paride Legovini]
+ - util: move uptime's else branch into its own boottime function (#53)
+   [Igor Galić] (LP: #1853160)
+ - workflows: add contributor license agreement checker (#155)
+ - net: fix rendering of 'static6' in network config (#77) (LP: #1850988)
+ - Make tests work with Python 3.8 (#139) [Conrad Hoffmann]
+ - fixed minor bug with mkswap in cc_disk_setup.py (#143) [andreaf74]
+ - freebsd: fix create_group() cmd (#146) [Gonéri Le Bouder]
+ - doc: make apt_update example consistent (#154)
+ - doc: add modules page toc with links (#153) (LP: #1852456)
+ - Add support for the amazon variant in cloud.cfg.tmpl (#119)
+   [Frederick Lefebvre]
+ - ci: remove Python 2.7 from CI runs (#137)
+ - modules: drop cc_snap_config config module (#134)
+ - migrate-lp-user-to-github: ensure Launchpad repo exists (#136)
+ - docs: add initial troubleshooting to FAQ (#104) [Joshua Powers]
+ - doc: update cc_set_hostname frequency and descrip (#109)
+   [Joshua Powers] (LP: #1827021)
+ - freebsd: introduce the freebsd renderer (#61) [Gonéri Le Bouder]
+ - cc_snappy: remove deprecated module (#127)
+ - HACKING.rst: clarify that everyone needs to do the LP->GH dance (#130)
+ - freebsd: cloudinit service requires devd (#132) [Gonéri Le Bouder]
+ - cloud-init: fix capitalisation of SSH (#126)
+ - doc: update cc_ssh clarify host and auth keys
+   [Joshua Powers] (LP: #1827021)
+ - ci: emit names of tests run in Travis (#120)
+
+19.4
+ - doc: specify _ over - in cloud config modules
+   [Joshua Powers] (LP: #1293254)
+ - tools: Detect python to use via env in migrate-lp-user-to-github
+   [Adam Dobrawy]
+ - Partially revert "fix unlocking method on FreeBSD" (#116)
+ - tests: mock uid when running as root (#113)
+   [Joshua Powers] (LP: #1856096)
+ - cloudinit/netinfo: remove unused getgateway (#111)
+ - docs: clear up apt config sections (#107) [Joshua Powers] (LP: #1832823)
+ - doc: add kernel command line option to user data (#105)
+   [Joshua Powers] (LP: #1846524)
+ - config/cloud.cfg.d: update README [Joshua Powers] (LP: #1855006)
+ - azure: avoid re-running cloud-init when instance-id is byte-swapped
+   (#84) [AOhassan]
+ - fix unlocking method on FreeBSD [Igor Galić] (LP: #1854594)
+ - debian: add reference to the manpages [Joshua Powers]
+ - ds_identify: if /sys is not available use dmidecode (#42)
+   [Igor Galić] (LP: #1852442)
+ - docs: add cloud-id manpage [Joshua Powers]
+ - docs: add cloud-init-per manpage [Joshua Powers]
+ - docs: add cloud-init manpage [Joshua Powers]
+ - docs: add additional details to per-instance/once [Joshua Powers]
+ - Update doc-requirements.txt [Joshua Powers]
+ - doc-requirements: add missing dep [Joshua Powers]
+ - dhcp: Support RedHat dhcp rfc3442 lease format for option 121 (#76)
+   [Eric Lafontaine] (LP: #1850642)
+ - network_state: handle empty v1 config (#45) (LP: #1852496)
+ - docs: Add document on how to report bugs [Joshua Powers]
+ - Add an Amazon distro in the redhat OS family [Frederick Lefebvre]
+ - removed a couple of "the"s [gaughen]
+ - docs: fix line length and remove highlighting [Joshua Powers]
+ - docs: Add security.md to readthedocs [Joshua Powers]
+ - Multiple file fix for AuthorizedKeysFile config (#60) [Eduardo Otubo]
+ - Revert "travis: only run CI on pull requests"
+ - doc: update links on README.md [Joshua Powers]
+ - doc: Updates to wording of README.md [Joshua Powers]
+ - Add security.md [Joshua Powers]
+ - setup.py: Amazon Linux sets libexec to /usr/libexec (#52)
+   [Frederick Lefebvre]
+ - Fix linting failure in test_url_helper (#83) [Eric Lafontaine]
+ - url_helper: read_file_or_url should pass headers param into readurl
+   (#66) (LP: #1854084)
+ - dmidecode: log result *after* stripping n [Igor Galić]
+ - cloud_tests: add azure platform support to integration tests
+   [ahosmanmsft]
+ - set_passwords: support for FreeBSD (#46) [Igor Galić]
+ - tools: migrate-lp-user-to-github removes repo_dir if created (#35)
+ - Correct jumbled documentation for cc_set_hostname module (#64)
+   [do3meli] (LP: #1853543)
+ - FreeBSD: fix for get_linux_distro() and lru_cache (#59)
+   [Igor Galić] (LP: #1815030)
+ - ec2: Add support for AWS IMDS v2 (session-oriented) (#55)
+ - tests: Fix cloudsigma tests when no dmidecode data is present. (#57)
+   [Scott Moser]
+ - net: IPv6, accept_ra, slaac, stateless (#51)
+   [Harald] (LP: #1806014, #1808647)
+ - docs: Update the configdrive datasource links (#44)
+   [Joshua Powers] (LP: #1852461)
+ - distro: correctly set usr_lib_exec path for FreeBSD distro (#40)
+   [Igor Galić] (LP: #1852491)
+ - azure: support secondary ipv6 addresses (#33)
+ - Fix metadata check when local-hostname is null (#32)
+   [Mark Goddard] (LP: #1852100)
+ - switch default FreeBSD salt minion pkg from py27 to py36
+   [Dominic Schlegel]
+ - travis: only run CI on pull requests
+ - add data-server dns entry as new metadata server detection [Joshua Hügli]
+ - pycodestyle: remove unused local variable
+ - reporting: Using a uuid to enforce uniqueness on the KVP keys. [momousta]
+ - docs: touchups in rtd intro and README.md
+ - doc: update launchpad git refs to github
+ - github: drop pull-request template to prepare for migration
+ - tools: add migrate-lp-user-to-github script to link LP to github
+ - github: new basic project readme
+
+19.3
+ - azure: support matching dhcp route-metrics for dual-stack ipv4 ipv6
+   (LP: #1850308)
+ - configdrive: fix subplatform config-drive for /config-drive source
+   [David Kindred] (LP: #1849731)
+ - DataSourceSmartOS: reconfigure network on each boot
+   [Mike Gerdts] (LP: #1765801)
+ - Add config for ssh-key import and consuming user-data [Pavel Zakharov]
+ - net: fix subnet_is_ipv6() for stateless|stateful
+   [Harald Jensås] (LP: #1848690)
+ - OVF: disable custom script execution by default [Xiaofeng Wang]
+ - cc_puppet: Implement csr_attributes.yaml support [Matthias Baur]
+ - cloud-init.service: on centos/fedora/redhat wait on NetworkManager.service
+   (LP: #1843334)
+ - azure: Do not lock user on instance id change [Sam Eiderman] (LP: #1849677)
+ - net/netplan: use ipv6-mtu key for specifying ipv6 mtu values
+ - Fix usages of yaml, and move yaml_dump to safeyaml.dumps. (LP: #1849640)
+ - exoscale: Increase url_max_wait to 120s. [Chris Glass]
+ - net/sysconfig: fix available check on SUSE distros
+   [Robert Schweikert] (LP: #1849378)
+ - docs: Fix incorrect Azure IMDS IP address [Joshua Powers] (LP: #1849508)
+ - introduce .travis.yml
+ - net: enable infiniband support in eni and sysconfig renderers
+   [Darren Birkett] (LP: #1847114)
+ - guestcust_util: handle special characters in config file [Xiaofeng Wang]
+ - fix some more typos in comments [Dominic Schlegel]
+ - replace any deprecated log.warn with log.warning
+   [Dominic Schlegel] (LP: #1508442)
+ - net: handle openstack dhcpv6-stateless configuration
+   [Harald Jensås] (LP: #1847517)
+ - Add .venv/ to .gitignore [Dominic Schlegel]
+ - Small typo fixes in code comments. [Dominic Schlegel]
+ - cloud_test/lxd: Retry container delete a few times
+ - Add Support for e24cloud to Ec2 datasource. (LP: #1696476)
+ - Add RbxCloud datasource [Adam Dobrawy]
+ - get_interfaces: don't exclude bridge and bond members (LP: #1846535)
+ - Add support for Arch Linux in render-cloudcfg [Conrad Hoffmann]
+ - util: json.dumps on python 2.7 will handle UnicodeDecodeError on binary
+   (LP: #1801364)
+ - debian/ubuntu: add missing word to netplan/ENI header (LP: #1845669)
+ - ovf: do not generate random instance-id for IMC customization path
+ - sysconfig: only write resolv.conf if network_state has DNS values
+   (LP: #1843634)
+ - sysconfig: use distro variant to check if available (LP: #1843584)
+ - systemd/cloud-init.service.tmpl: start after wicked.service
+   [Robert Schweikert]
+ - docs: fix zstack documentation lints
+ - analyze/show: remove trailing space in output
+ - Add missing space in warning: "not avalid seed" [Brian Candler]
+ - pylintrc: add 'enter_context' to generated-members list
+ - Add datasource for ZStack platform. [Shixin Ruan] (LP: #1841181)
+ - docs: organize TOC and update summary of project [Joshua Powers]
+ - tools: make clean now cleans the dev directory, not the system
+ - docs: create cli specific page [Joshua Powers]
+ - docs: added output examples to analyze.rst [Joshua Powers]
+ - docs: doc8 fixes for instancedata page [Joshua Powers]
+ - docs: clean up formatting, organize boot page [Joshua Powers]
+ - net: add is_master check for filtering device list (LP: #1844191)
+ - docs: more complete list of availability [Joshua Powers]
+ - docs: start FAQ page [Joshua Powers]
+ - docs: cleanup output & order of datasource page [Joshua Powers]
+ - Brightbox: restrict detection to require full domain match .brightbox.com
+ - VMWware: add option into VMTools config to enable/disable custom script.
+   [Xiaofeng Wang]
+ - net,Oracle: Add support for netfailover detection
+ - atomic_helper: add DEBUG logging to write_file (LP: #1843276)
+ - doc: document doc, create makefile and tox target [Joshua Powers]
+ - .gitignore: ignore files produced by package builds
+ - docs: fix whitespace, spelling, and line length [Joshua Powers]
+ - docs: remove unnecessary file in doc directory [Joshua Powers]
+ - Oracle: Render secondary vnic IP and MTU values only
+ - exoscale: fix sysconfig cloud_config_modules overrides (LP: #1841454)
+ - net/cmdline: refactor to allow multiple initramfs network config sources
+ - ubuntu-drivers: call db_x_loadtemplatefile to accept NVIDIA EULA
+   (LP: #1840080)
+ - Add missing #cloud-config comment on first example in documentation.
+   [Florian Müller]
+ - ubuntu-drivers: emit latelink=true debconf to accept nvidia eula
+   (LP: #1840080)
+ - DataSourceOracle: prefer DS network config over initramfs
+ - format.rst: add text/jinja2 to list of content types (+ cleanups)
+ - Add GitHub pull request template to point people at hacking doc
+ - cloudinit/distros/parsers/sys_conf: add docstring to SysConf
+ - pyflakes: remove unused variable [Joshua Powers]
+ - Azure: Record boot timestamps, system information, and diagnostic events
+   [Anh Vo]
+ - DataSourceOracle: configure secondary NICs on Virtual Machines
+ - distros: fix confusing variable names
+ - azure/net: generate_fallback_nic emits network v2 config instead of v1
+ - Add support for publishing host keys to GCE guest attributes [Rick Wright]
+ - New data source for the Exoscale.com cloud platform [Chris Glass]
+ - doc: remove intersphinx extension
+ - cc_set_passwords: rewrite documentation (LP: #1838794)
+ - net/cmdline: split interfaces_by_mac and init network config determination
+ - stages: allow data sources to override network config source order
+ - cloud_tests: updates and fixes
+ - Fix bug rendering MTU on bond or vlan when input was netplan. (LP: #1836949)
+ - net: update net sequence, include wait on netdevs, opensuse netrules path
+   (LP: #1817368)
+19.2:
+ - net: add rfc3442 (classless static routes) to EphemeralDHCP
+   (LP: #1821102)
+ - templates/ntp.conf.debian.tmpl: fix missing newline for pools
+   (LP: #1836598)
+ - Support netplan renderer in Arch Linux [Conrad Hoffmann]
+ - Fix typo in publicly viewable documentation. [David Medberry]
+ - Add a cdrom size checker for OVF ds to ds-identify
+   [Pengpeng Sun] (LP: #1806701)
+ - VMWare: Trigger the post customization script via cc_scripts module.
+   [Xiaofeng Wang] (LP: #1833192)
+ - Cloud-init analyze module: Added ability to analyze boot events.
+   [Sam Gilson]
+ - Update debian eni network configuration location, retain Ubuntu setting
+   [Janos Lenart]
+ - net: skip bond interfaces in get_interfaces
+   [Stanislav Makar] (LP: #1812857)
+ - Fix a couple of issues raised by a coverity scan
+ - Add missing dsname for Hetzner Cloud datasource [Markus Schade]
+ - doc: indicate that netplan is default in Ubuntu now
+ - azure: add region and AZ properties from imds compute location metadata
+ - sysconfig: support more bonding options [Penghui Liao]
+ - cloud-init-generator: use libexec path to ds-identify on redhat systems
+   (LP: #1833264)
+ - tools/build-on-freebsd: update to python3 [Gonéri Le Bouder]
+ - Allow identification of OpenStack by Asset Tag
+   [Mark T. Voelker] (LP: #1669875)
+ - Fix spelling error making 'an Ubuntu' consistent. [Brian Murray]
+ - run-container: centos: comment out the repo mirrorlist [Paride Legovini]
+ - netplan: update netplan key mappings for gratuitous-arp (LP: #1827238)
+ - freebsd: fix the name of cloudcfg VARIANT [Gonéri Le Bouder]
+ - freebsd: ability to grow root file system [Gonéri Le Bouder]
+ - freebsd: NoCloud data source support [Gonéri Le Bouder] (LP: #1645824)
+ - Azure: Return static fallback address as if failed to find endpoint
+   [Jason Zions (MSFT)]
+
+19.1:
+  - freebsd: add chpasswd pkg in the image [Gonéri Le Bouder]
+  - tests: add Eoan release [Paride Legovini]
+  - cc_mounts: check if mount -a on no-change fstab path
+    [Jason Zions (MSFT)] (LP: #1825596)
+  - replace remaining occurrences of LOG.warn [Daniel Watkins]
+  - DataSourceAzure: Adjust timeout for polling IMDS [Anh Vo]
+  - Azure: Changes to the Hyper-V KVP Reporter [Anh Vo]
+  - git tests: no longer show warning about safe yaml.
+  - tools/read-version: handle errors [Chad Miller]
+  - net/sysconfig: only indicate available on known sysconfig distros
+    (LP: #1819994)
+  - packages: update rpm specs for new bash completion path
+    [Daniel Watkins] (LP: #1825444)
+  - test_azure: mock util.SeLinuxGuard where needed
+    [Jason Zions (MSFT)] (LP: #1825253)
+  - setup.py: install bash completion script in new location [Daniel Watkins]
+  - mount_cb: do not pass sync and rw options to mount
+    [Gonéri Le Bouder] (LP: #1645824)
+  - cc_apt_configure: fix typo in apt documentation [Dominic Schlegel]
+  - Revert "DataSource: move update_events from a class to an instance..."
+    [Daniel Watkins]
+  - Change DataSourceNoCloud to ignore file system label's case.
+    [Risto Oikarinen]
+  - cmd:main.py: Fix missing 'modules-init' key in modes dict
+    [Antonio Romito] (LP: #1815109)
+  - ubuntu_advantage: rewrite cloud-config module
+  - Azure: Treat _unset network configuration as if it were absent
+    [Jason Zions (MSFT)] (LP: #1823084)
+  - DatasourceAzure: add additional logging for azure datasource [Anh Vo]
+  - cloud_tests: fix apt_pipelining test-cases
+  - Azure: Ensure platform random_seed is always serializable as JSON.
+    [Jason Zions (MSFT)]
+  - net/sysconfig: write out SUSE-compatible IPv6 config [Robert Schweikert]
+  - tox: Update testenv for openSUSE Leap to 15.0 [Thomas Bechtold]
+  - net: Fix ipv6 static routes when using eni renderer
+    [Raphael Glon] (LP: #1818669)
+  - Add ubuntu_drivers config module [Daniel Watkins]
+  - doc: Refresh Azure walinuxagent docs [Daniel Watkins]
+  - tox: bump pylint version to latest (2.3.1) [Daniel Watkins]
+  - DataSource: move update_events from a class to an instance attribute
+    [Daniel Watkins] (LP: #1819913)
+  - net/sysconfig: Handle default route setup for dhcp configured NICs
+    [Robert Schweikert] (LP: #1812117)
+  - DataSourceEc2: update RELEASE_BLOCKER to be more accurate
+    [Daniel Watkins]
+  - cloud-init-per: POSIX sh does not support string subst, use sed
+    (LP: #1819222)
+  - Support locking user with usermod if passwd is not available.
+  - Example for Microsoft Azure data disk added. [Anton Olifir]
+  - clean: correctly determine the path for excluding seed directory
+    [Daniel Watkins] (LP: #1818571)
+  - helpers/openstack: Treat unknown link types as physical
+    [Daniel Watkins] (LP: #1639263)
+  - drop Python 2.6 support and our NIH version detection [Daniel Watkins]
+  - tip-pylint: Fix assignment-from-return-none errors
+  - net: append type:dhcp[46] only if dhcp[46] is True in v2 netconfig
+    [Kurt Stieger] (LP: #1818032)
+  - cc_apt_pipelining: stop disabling pipelining by default
+    [Daniel Watkins] (LP: #1794982)
+  - tests: fix some slow tests and some leaking state [Daniel Watkins]
+  - util: don't determine string_types ourselves [Daniel Watkins]
+  - cc_rsyslog: Escape possible nested set [Daniel Watkins] (LP: #1816967)
+  - Enable encrypted_data_bag_secret support for Chef
+    [Eric Williams] (LP: #1817082)
+  - azure: Filter list of ssh keys pulled from fabric [Jason Zions (MSFT)]
+  - doc: update merging doc with fixes and some additional details/examples
+  - tests: integration test failure summary to use traceback if empty error
+  - This is to fix https://bugs.launchpad.net/cloud-init/+bug/1812676
+    [Vitaly Kuznetsov]
+  - EC2: Rewrite network config on AWS Classic instances every boot
+    [Guilherme G. Piccoli] (LP: #1802073)
+  - netinfo: Adjust ifconfig output parsing for FreeBSD ipv6 entries
+    (LP: #1779672)
+  - netplan: Don't render yaml aliases when dumping netplan (LP: #1815051)
+  - add PyCharm IDE .idea/ path to .gitignore [Dominic Schlegel]
+  - correct grammar issue in instance metadata documentation
+    [Dominic Schlegel] (LP: #1802188)
+  - clean: cloud-init clean should not trace when run from within cloud_dir
+    (LP: #1795508)
+  - Resolve flake8 comparison and pycodestyle over-ident issues
+    [Paride Legovini]
+  - opennebula: also exclude epochseconds from changed environment vars
+    (LP: #1813641)
+  - systemd: Render generator from template to account for system
+    differences. [Robert Schweikert]
+  - sysconfig: On SUSE, use STARTMODE instead of ONBOOT
+    [Robert Schweikert] (LP: #1799540)
+  - flake8: use ==/!= to compare str, bytes, and int literals
+    [Paride Legovini]
+  - opennebula: exclude EPOCHREALTIME as known bash env variable with a
+    delta (LP: #1813383)
+  - tox: fix disco httpretty dependencies for py37 (LP: #1813361)
+  - run-container: uncomment baseurl in yum.repos.d/*.repo when using a
+    proxy [Paride Legovini]
+  - lxd: install zfs-linux instead of zfs meta package
+    [Johnson Shi] (LP: #1799779)
+  - net/sysconfig: do not write a resolv.conf file with only the header.
+    [Robert Schweikert]
+  - net: Make sysconfig renderer compatible with Network Manager.
+    [Eduardo Otubo]
+  - cc_set_passwords: Fix regex when parsing hashed passwords
+    [Marlin Cremers] (LP: #1811446)
+  - net: Wait for dhclient to daemonize before reading lease file
+    [Jason Zions] (LP: #1794399)
+  - [Azure] Increase retries when talking to Wireserver during metadata walk
+    [Jason Zions]
+  - Add documentation on adding a datasource.
+  - doc: clean up some datasource documentation.
+  - ds-identify: fix wrong variable name in ovf_vmware_transport_guestinfo.
+  - Scaleway: Support ssh keys provided inside an instance tag. [PORTE Loïc]
+  - OVF: simplify expected return values of transport functions.
+  - Vmware: Add support for the com.vmware.guestInfo OVF transport.
+    (LP: #1807466)
+  - HACKING.rst: change contact info to Josh Powers
+  - Update to pylint 2.2.2.
+
 18.5:
  - tests: add Disco release [Joshua Powers]
  - net: render 'metric' values in per-subnet routes (LP: #1805871)