| Age | Commit message (Collapse) | Author |
|---|
|
|
|
The result of a read_file_or_url on a file and on a url would differ
in behavior.
str(UrlResponse) would return UrlResponse.contents.decode('utf-8')
while
str(FileResponse) would return str(FileResponse.contents)
The difference being "b'foo'" versus "foo".
As part of the general goal of cleaning util, move read_file_or_url
into url_helper.
|
|
pylint missed finding a typo in the lxd platform because it could not
determine that the variable was being used was a string. The variable
was set by loading a yaml file which pylint couldn't know that it
would be a string. In these cases, we can be more explicit.
|
|
|
|
This also makes some of the messages more consistent.
|
|
The SSH function was retrying and waiting for SSH for over an
hour when an SSH connection was failing to be established. This
reduces the amount of retries and time between each retry to
prevent tests from running for hours.
Also restructures how waiting for the system works: the system
will attempt to SSH up to the boot timeout time by catching
SSH connection failures and retrying until the timeout is
reached. If the limit is reached now an exception is thrown
to abort the test.
Drive by - this also fixes printing of the instance name when
collecting the console log, rather than showing a Python object
address.
Fixes LP: #1758409
|
|
In playing with a SmartOS container I found that ds-identify did
not identify the container there as a container. Systemd-detect-virt
identifies it as 'container-other'.
Also here are tests for ds-identify for the SmartOS platform
identification, and some indentation fixes in ds-identify.
|
|
This makes cloud-config.service (and as a result cloud-final.service)
run After snap.seeded.service. This is required to ensure that
pre-seeded snaps can be used by cloud-init or user-data input.
The snap.seeded.service was added to snapd at:
https://github.com/snapcore/snapd/pull/5124
Note that the following would be a workaround:
snap:
commands:
00: snap wait system seed.loaded
LP: #1767131
|
|
Make test_net.TestGenerateFallbackConfig.test_unstable_names mock
the value of /proc/cmdline in the same way as the existing
test_unstable_names_disabled test.
LP: #1769952
|
|
We had two calls to is_ds_enabled, and the debug message looked
something like this:
is_ds_enabled returned 1: ConfigDrive NoCloud
Now instead we have just one call, and the debug message like:
is_ds_enabled(IBMCloud) = true
|
|
When attempting to apply network configuration for SmartOS's container
platform, cloud-init would not identify nics. The nics on provided
in this container service do not have 'addr_assign_type'. That
was being interpreted as being a "stolen" mac, and would be filtered
out by get_interfaces.
|
|
package_update_upgrade_install was failing as htop is now included in
Bionic images. Switch this test to install 'sl' instead.
ca_certs integration test fails on cert_count test because bionic
update-ca-certificates on bionic generates less symlinks for a given cert.
Integration tests now collect dpkg-query --show output on every instance.
Add a new assertPackageInstalled helper method which finds the package or
package version installed on the instance.
Adapt existing byobu, package_update_upgrade_install, ntp and salt_minion
tests to use assertPackageInstalled method.
LP: #1769985
|
|
This fixes warnings reported by shellcheck at 0.4.6.
The complaints that we are ignoring globally (top of the file) are:
2015: Note that A && B || C is not if-then-else. C may run if A is true.
2039: In POSIX sh, 'local' is undefined.
2162: read without -r will mangle backslashes.
2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.
Most of the complaints were just noise, but a few unused variables
were reported and fixed.
Related shellcheck issues opened:
- https://github.com/koalaman/shellcheck/issues/1191
- https://github.com/koalaman/shellcheck/issues/1192
- https://github.com/koalaman/shellcheck/issues/1193
- https://github.com/koalaman/shellcheck/issues/1194
|
|
Fix remaining pycodesytle warnings related to invalid string literals
introduced in more recent pycodeflakes versions
https://bugs.python.org/issue27364 .
Also stop using flake8 in tox as it is incompatible with newer versions of
pyflakes. Instead we now add tox environments for pycodestyle and pyflakes
individually.
Set the versions in both pycodestyle and pyflakes to the currently
available versions.
|
|
This change is for Azure VM Preprovisioning. A bug was found when after
azure VMs report ready the first time, during the time when VM is polling
indefinitely for the new ovf-env.xml from Instance Metadata Service
(IMDS), if a reboot happens, we send another report ready signal to the
fabric, which deletes the reprovisioning data on the node.
This marker file is used to fix this issue so that we will only send a
report ready signal to the fabric when no marker file is present. Then,
create a marker file so that when a reboot does occur, we check if a
marker file has been created and decide whether we would like to send the
repot ready signal.
LP: #1765214
|
|
bddeb already supported passing in a '--release' and that would get
into the changelog line.
If you used bddeb to build packages for a PPA, and built multiple
releases, then you would get the same version for each release, and
launchpad would reject your upload.
The change here means we get a ~16.04.1 (for xenial) suffix on the
dpkg version. If the distro-info-data package is not installed,
or the release is not known (such as the default "UNRELEASED"),
then you get no suffix.
|
|
By default, FreeBSD's growfs runs interactively asking a question
which can be mitigated using the '-y' command line option. The fix
here is simply to pass -y to growfs to avoid the prompt.
LP: #1404745
|
|
If you built packages with 'bddeb', each time it would create a new
tarball with make-tarball. If you then tried to upload two different
tarballs to launchpad (to a PPA), it would reject the second as the
orig tarball already existed.
This just supports looking in some places for a orig tarball and
re-using if it is found.
|
|
The last set of changes to netdev_pformat ended up dropping the output
of devices that were not up. This adds back the 'down' interfaces to the
rendered output.
LP: #1766302
|
|
With no output at all from collect-logs, users have been confused
on where the output is. By default now, write to stderr what that
file is.
Also
* add '-v' to increase verbosity. With a single -v flag, mention
what file/info is being collected.
* limit the 'journalctl' collection to this boot (--boot=0).
collecting entire journal seems unnecessary and can be huge.
* do not fail when collecting files or directories that are not there.
LP: #1766335
|
|
Ubuntu images on IBMCloud for 16.04 have some seed data in
/var/lib/cloud/data/seed/nocloud-net. In order to have systems with
IBMCloud enabled, we modified ds-identify detection to skip that seed
if the system was on IBMCloud. That change did not consider the
fact that IBMCloud might not be in the datasource list.
There was similar logic in the ConfigDrive datasource in ds-identify
and the datasource itself.
Config drive is now updated to only check and avoid IBMCloud if IBMCloud
is enabled. The check in ds-identify for nocloud was dropped. If a
user provides a nocloud seed on IBMCloud, then that can be used.
This means that systems running Xenial will continue to get their
old datasources.
LP: #1766401
|
|
In looking at some boot time for Xenial, Artful and Bionic, we noticed
some long amounts of time that appeared to be part of the DataSource but
we related to resolving URLs. In Artful and Bionic, there was an issue
(bug: #1739672) that resulted in slow getaddrinfo() calls when
systemd-resolved was in use. This patch adds two events that track time
for datasource.setup_datasource() and datasource.activate_datasource()
Additionally use log_time() to wrapper util.is_resolvable_url() which
leaves info in cloud-init.log about how much time was spent.
|
|
When images are deployed from template in a production environment
the artifacts of the provisioning stage (provisioningConfiguration.cfg)
that cloud-init referenced are cleaned up. However, when provisioned
in "debug" mode (internal to IBM) the artifacts are left.
This changes the 'is_ibm_provisioning' implementations in both
ds-identify and in the IBM datasource to identify the provisioning
stage more correctly. The change is to consider provisioning only
if the provisioing file existed and there was no log file or
the log file was older than this boot.
LP: #1767166
|
|
The cloud-init-local.service expects that any network device name changes
have already been completed by the kernel or udev daemon.
In some situations we've found that the renaming of interfaces from kernel
names (eth0, eth1, etc) to their persistent names (eno1, ens3, enp0s1,
etc) may happen after cloud-init-local has started where it reads values
from sysfs about what network devices are present, and which device to use
as a fallback nic.
Subsequently, cloud-init-local would write out network configuration for a
kernel device name which would no longer be present by the time that
networking services start to bring up the devices. The result is that the
instance does not get networking configured. Prior to use of
systemd-networkd, the Ubuntu 'networking.service' unit included a call to
udevadm settle which is why this race is not seen on a Xenial system.
This change adds the ability to detect if an interface has a stable name,
if if we find one without stable names and stable names have not been
disabled (net.ifnames=0 in /proc/cmdline), then cloud-init will invoke
udevadm settle.
LP: #1766287
|
|
This adds information to the IBMCloud datasource describing the
6 different scenarios that it is expected to handle.
|
|
BOOTPROTO=dhcp in sysconfig enables DHCPv4 and we should not do this
implicitly when 'dhcp6' subnet is specified. In case both dhcpv4 and
dhcpv6 are needed users should specify both:
subnets:
- type: dhcp6
- type: dhcp
Fix the current code and add a dhcpv6 only test.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
|
|
Ubuntu minimal images do not have iproute2, so correctly identify
our dependency on it.
LP: #1766711
|
|
cloud-init and mdata-get each have their own implementation of the
SmartOS metadata protocol. If cloud-init and other services that call
mdata-get are run concurrently, crosstalk on the serial port can cause
them both to become confused.
This change makes it so that cloud-init uses the same cooperative
locking scheme that's used by mdata-get, thus preventing cross-talk
between mdata-get and cloud-init.
For testing, a VM running on a SmartOS host and pyserial are required.
If the tests are run on a platform other than SmartOS, those that use a
real serial port are skipped. pyserial remains commented in
requirements.txt because most testers will not be running atop SmartOS.
LP: #1746605
|
|
There are three potential sources of the hostname, one of which is
documented SmartOS's vmadm(1M) via the hostname property. That
property's value is retrieved via the sdc:hostname key. The other
two sources for the hostname are a hostname key in customer_metadata
and the VM's uuid (sdc:uuid). Of these three, the sdc:hostname value
is not used in a meaningful way by DataSourceSmartOS.
This fix changes the fallback mechanism when hostname is not
specified in customer_metadata. The order of precedence for setting
the hostname is now 1) hostname in customer_metadata,
2) sdc:hostname, then 3) sdc:uuid.
LP: #1765085
|
|
If customer_metadata has no keys, the KEYS request returns an empty
string. Callers of the list() method expect a list to be returned and
will give a stack trace if this expectation is not met.
LP: #1763480
|
|
validate_cloudconfig_schema with strict=True would not actually validate
if there was no jsonschema available. That seems kind of strange.
The change here is to make it raise an exception if strict was passed in.
And then to fix the one test that needed a skipIfJsonSchema wrapper.
|
|
This admittedly does a fairly extensive re-factor to simply add a newline
to the end of sshd_config.
It makes the ssh_config updating portion of set_passwords more testable
and adds tests for that.
The new function is in 'update_ssh_config_lines' which allows you
to update a config with multiple changes even though only a single one
is currently used.
We also only restart the ssh daemon now if a change was made to the
config file. Before it was always restarted if the user specified
a value for ssh_pwauth other than 'unchanged'.
Thanks to Lorens Kockum for initial diagnosis and patch.
LP: #1677205
|
|
This enables warnings produced by pylint for unused variables (W0612),
and fixes the existing errors.
|
|
Just add some documentation to readthedocs for AliYun.
|
|
runcmd, bootcmd, snap/commands, ubuntu-advantage/commands would
log warning (and fail if strict) on duplicate values in the commands.
But those should be allowed. Example, it is perfectly valid to do:
runcmd: ['sleep 1', 'sleep 1']
LP: #1764264
|
|
The net-tools package is deprecated and will eventually be dropped. Use
"ip route", "link" or "address" instead of "ifconfig" or "route" calls.
Cloud-init can now run in an environment that no longer has net-tools.
This affects the network and route printing emitted to
cloud-config-output.log as well as the cc_disable_ec2_metadata module.
Additional changes:
- separate readResource and resourceLocation into standalone test
functions
- Fix ipv4 address rows to report scopes represented by ip addr show
- Formatted route/address ouput now handles multiple ipv4 and ipv6
addresses on a single interface
Co-authored-by: James Hogarth <james.hogarth@gmail.com>
Co-authored-by: Robert Schweikert <rjschwei@suse.com>
|
|
If the metadata service in the host is down while a guest that uses
DataSourceSmartOS is booting, the request from the guest falls into the
bit bucket. When the metadata service is eventually started, the guest
has no awareness of this and does not resend the request. This results in
cloud-init hanging forever with a guest reboot as the only recovery
option.
This fix updates the metadata protocol to implement the initialization
phase, just as is implemented by mdata-get and related utilities. The
initialization phase includes draining all pending data from the serial
port, writing an empty command and getting an expected error message in
reply. If the initialization phase times out, it is retried every five
seconds. Each timeout results in a warning message: "Timeout while
initializing metadata client. Is the host metadata service running?" By
default, warning messages are logged to the console, thus the reason for a
hung boot is readily apparent.
LP: #1667735
|
|
ext3 is not able to support file system sizes that are needed in Joyent's
cloud. For the default block size of 4k, the maximum filesystem size
for ext3 is 2^32 * 4096 = 16 TiB.
This changes the default file system type from ext3 to ext4.
LP: #1763511
|
|
Python has deprecated these invalid string literals now
https://bugs.python.org/issue27364
and pycodestyle is identifying them with a W605 warning.
https://github.com/PyCQA/pycodestyle/pull/676
So basically, any use of \ not followed by one of [\'"abfnrtv]
or \ooo (octal) \xhh (hex) or a newline is invalid. This is most
comomnly seen for us in regex. To solve, you either:
a.) use a raw string r'...'
b.) correctly escape the \ that was not intended to be interpreted.
|
|
In bash shells with bash_completion enabled, now the cloud-init
sub commands and parameters/flags will be shown.
|
|
This tool is used to assist during the creation of ubuntu packages for
release testing. Address the following on the command-line:
* --help option now print usage
* Add --orig-tarball which creates named output file
cloud-init_<release-version>.orig.tar.gz
* drop unused --verbose option
|
|
If a file passed to render_from_file had non-ascii text then
jinja in python2 would decode as ascii, which would cause
UnicodeDecodeError. This issue can be re-created in python2
with just:
'can\xe2\x80\x99t'.decode()
The solution here is to explicitly pass in unicode supporting
type (py3 str, py2 unicode). Those are six.text_type.
Then jinja does not try to decode.
The reason we hit this is that load_file calls decode_binary.
decode_binary believes it has no work to do if it got a six.string_types.
isinstance('can\xe2\x80\x99t', six.string_types) == True
So it returns the original string which will blow up for jinja.
Our fix here then is to load the file in binary mode and explicitly
decode it to utf-8. Then in python2 we'll have a unicode type
and in python3 we'll have a string type.
|
|
Add a base NTP client configuration dictionary and allow Distro
specific changes to be merged. Add a select client function which
implements logic to preferr installed clients over clients which
need to be installed. Also allow distributions to override the
cloud-init defaults.
LP: #1749722
|
|
When filing a bug with apport, this allows the user to choose
Brightbox, IBM, LXD, or OpenTelekomCloud as their cloud.
|
|
Fix integraiton test logic for ec2 to look for network and
availability-zone data under the key path
'ds'=>'meta-data' instead of just 'ds' when parsing instance-data.json.
|
|
Integration tests previously had a logic path that was unexercised on
jenkins because we were on an older version of lxc. With an upgrade to lxd
version 3.0 we need to bump pylxd dependency pin and fix a typo in
integration tests which checked the lxd version.
|
|
LP: #1420018
|
|
The zfs/zpool commands will hang for 10 seconds if /dev/zfs is not
present (bug 1760173). This is a common occurence for containers
using zfs as rootfs. Additionally handle missing zpool command or
other errors that may occur while executing the zpool command.
|
|
Fix link to external openstack resource and to internal vendor data.
LP: #1721660
|
|
Bump the version in cloudinit/version.py to be 18.2 and update ChangeLog.
LP: #1759318
|
