| Age | Commit message (Collapse) | Author |
|---|
|
This just moves flake8 and related tools up to newer versions and fixes
the complaints associated with that.
We added to the list of flake8 ignores:
H102: do not put vim info in source files
H304: no relative imports
Also updates and pins the following in the flake8 environment:
pep8: 1.7.0 => drop (although hacking still pulls it in).
pyflakes 1.1.0 => 1.5.0
hacking 0.10.2 => 0.13.0
flake8 2.5.4 => 3.3.0
pycodestyle none => 2.3.1
|
|
Dual stack IPv4/IPv6 configuration via config drive is broken for RHEL7.
This patch fixes several scenarios for IPv4/IPv6/dual-stack with multiple
IP assignment.
Removes usage of unpopular IPv4 alias files and invalid IPv6 alias files.
Also fix associated unit tests.
LP: #1679817
LP: #1685534
LP: #1685532
|
|
This fixes several shortcomings of disk_setup with gpt disks.
* 'sgdisk -p' was being used to determine the size of a disk.
this can fail if it believes there is a bad gpt partition table.
Instead we just use blockdev now for both mbr or gpt disks.
* parsing of sgdisk -p output assumed that the 'name' of the partition
type would not have any spaces (Microsoft basic data)
* interaction with sgdisk did not realize that sgdisk wants input
of '8300' rather than '83' and will output the same.
LP: #1692087
|
|
Some interfaces (greptap0 in the bug) have a mac address of
'00:00:00:00:00:00'. That was causing a duplicate mac detection
as the 'lo' device also has that mac.
The change here is to just ignore macs other than 'lo' that have that.
LP: #1692028
|
|
Previously, the datasource for DigitalOcean allowed for a gateway on each
NIC. As a result, on Ubuntu 16.04, networking.service was broken. For
17.04 and later, Ubuntu _replaces_ the default gateway with the second
gateway on 'ifup' after reboot.
DigitalOcean is looking at changing the meta-data, however, this will
result in another version of the meta-data JSON.
LP: #1681531.
|
|
When vlan mac address is specified in config, render it for
netplan and for ENI.
LP: #1690388
|
|
On systems which installed ntp and specified servers or pools in the
config ntpd didn't notice the updated configuration file and didn't
use the correct configuration. Resolve this by rendering the template
first which allows the package install to use the existing
configuration. Additionally add a service restart to handle the case
where ntp does not need to be installed but it may not have started.
Add an integration test to confirm that cc_ntp enables ntp to use the
specific servers and pools in the cloud-config.
LP: #1645644
|
|
on centos/fedora/rhel, /var/lib/NetworkManager has mode 700, causing
the cloudstack unit tests to fail when run as a non-root user. This
mocks out get_latest_lease so that we no longer try to read dhcp lease
information during the unit tests.
|
|
A recent merge that added a mkfs.ext4 tests has a hard coded location
for the binary of mkfs.ext4. On CentOS 7 the test failed because the
command in a different location than Ubuntu.
LP: #1691517
|
|
tools/net-convert fails to output netplan config, because the
positional arguments of render_network_state are the wrong way around
for that function w.r.t. other renders.
Fix the netplan renderer to have the correct signature.
LP: #1685944
|
|
Large instance types have a different disk format on the newly
partitioned ephemeral drive. So we have to adjust the logic in the
Azure datasource to recognize that a disk with 2 partitions and
an empty ntfs filesystem on the second one is acceptable.
This also adjusts the datasources's builtin fs_setup config to remove
the 'replace_fs' entry. This entry was previously ignored, and confusing.
I've clarified the doc on that also.
LP: #1686514
|
|
Apt related tests were broken when running on centos becasue apt is not
available. This fixes the unit test, with a small re-work of apt_configure.
Also in 'tox -e centos6' only run nose on tests/unittests as tests/
also contain integration tests that should not be run.
|
|
Recent core snap images (edge channel revision 1886) do not contain the
previously known files used to detect that a system is ubuntu core.
The changes here are to look in 2 additional locations to determine
if a system is snappy.
LP: #1689944
|
|
If /run/net-<name>.cfg contains an IPV4ADDR or an IPV6ADDR, the config
file generated by _klibc_to_config_entry now contains the "address".
LP: #1691135
|
|
Fixed setting Route.has_set_default_ipv6 or *_ipv4 to track whether a
route already has a default gateway defined. The code was setting
Route.has_set_default which wasn't checked when raising "duplicate
gateway" ValueErrors. Added unit tests to exercise this expected raised
ValueError. Also moved is_ipv6 = subnet.get('ipv6') logic out of a for
loop because we don't need to recalculate the same value every route
iteration.
LP: #1687485
|
|
This patch targets to make FreeBSD 10.3 or 11 work on Azure. The
modifications abide by the rule of:
* making as less modification as possible
* delegate to the distro or datasource where possible.
The main modifications are:
1. network configuration improvements, and movement into distro path.
2. Fix setting of password.
Password setting through "pw" can only work through pipe.
3. Add 'root:wheel' to syslog_fix_perms field.
4. Support resizing default file system (ufs)
5. copy cloud.cfg for freebsd to /etc/cloud/cloud.cfg rather than
/usr/local/etc/cloud/cloud.cfg.
6. Azure specific changes:
a. When reading the azure endpoint, search in a different path
and read a different option name (option-245 vs. unknown-245).
so, the lease file path should be generated according to platform.
b. adjust the handling of ephemeral mounts for ufs filesystem and
for finding the ephemeral device.
c. fix mounting of cdrom
LP: #1636345
|
|
This adds several unit tests for ds-identify, and fixes a bug
in Ec2 detection that I found while writing these tests.
The method of testing is to use the ds-identify code as a shell
library. The TestDsIdentify:call basically does:
* populate a (temp) directory with files that represent what
ds-identify would see in /sys or other locations it reads.
* create a file '_shwrap' that replaces the 3 programs that are executed
in ds-identify code path. It supports setting their stdout, stderr,
and exit code.
* set the default policies explicitly (DI_DEFAULT_POLICY) so we can
support testing different builtins. This is necessary because the
Ubuntu branches patch the builtin value. If we did not explicilty set
it, then testing there would fail.
* execute sh to source the script and call its main.
|
|
If 'cmd' is provided to a fs_setup entry, then cloud-init was trying
to execute the rendered string as a single name, rather than
splitting the string. The change here will pass the string to
shell for interpretation so that it is split there.
Also fix some documentation errors and warn when fs_opts or overwrite
is provided along with 'cmd'.
LP: #1687712
|
|
ConfigObj produces configuration files that are incompatible with yum if
multiple values are listed for a configuration key. Switch to the builtin
configparser, and ConfigParser (Python 2) which correctly handles this
case.
Add additional test case for array values in yum_repos definition
LP: #1592150
|
|
Introduce is_vlan function and call that when building dictionary of
interfaces by mac address.
LP: #1682871
|
|
On centos/fedora/rhel/derivatives, /etc/ssh/sshd_config has mode 0600,
but cloud-init unilaterally sets file modes to 0644 when no explicit
mode is passed to util.write_file. On ubuntu/debian, this file has
mode 0644. With this patch, write_file learns about the copy_mode
option, which will cause it to use the mode of the existing file by
default, falling back to the explicit mode parameter if the file does
not exist.
LP: #1644064
Resolves: rhbz#1295984
|
|
This change makes the DigitalOcean datasource consistent with OpenStack and
Joyent by binding the resolver addresses to the loopback interface. This _is_
a work-around to bug 1675571.
Part of bug 1676908.
|
|
This just mocks out the AltCloud tests to not invoke blkid.
Our tests should not rely on system command returning any specific
value.
Also, shorten long lines with change in the import name of
DataSourceAltCloud.
LP: #1636531
|
|
When cloud-init ran in the init stage (after networking had come up).
A bug could occur where cloud-init would attempt and fail to rename
network devices that had "inherited" mac addresses.
The intent of apply_network_config_names was always to rename only
the devices that were "physical" per the network config. (This would
include veth devices in a container). The bug was in creating
the dictionary of interfaces by mac address. If there were multiple
interfaces with the same mac address then renames could fail.
This situation was guaranteed to occur with bonds or vlans or other
devices that inherit their mac.
The solution is to change get_interfaces_by_mac to skip interfaces
that have an inherited mac.
Also drop the 'devs' argument to get_interfaces_by_mac. It was
non-obvious what the result should be if a device in the input
list was filtered out. ie should the following have an entry for
bond0 or not. get_interfaces_by_mac(devs=['bond0'])
LP: #1669860
|
|
Mock the use use of get_interfaces_by_mac in Digital Ocean and OpenNebula.
Its best to mock this for the tests as the results aren't expecting
it to fail.
Note, as it stands, OpenNebula relies on devices named 'eth0'.
The metadata (context) does not provide mac addresses.
|
|
When rendering network configuration to netplan, remove known
"builtin" configurations. The specific example here is Ubuntu Core
that has netplan configuration in etc/netplan/00-snapd-config.yaml.
We also delete the derived files since netplan will have created
these derived files in its generator that runs well before cloud-init.
LP: #1675576
|
|
While documentation indicates that the smbios product name should
contain 'Google Compute Engine', experimentation and bug reports
indicate that is not always the case. The change here is to change
the check for GCE to also consider a serial number that starts with
'GoogleCompute-'.
Also, ds-identify was not currently searching for GCE if no config of
datasource_list was found. Most images have a datasource_list defined.
So update the list to include GCE.
LP: #1674861
|
|
Accordingly to the documentation:
The ``partition`` option may also be set to ``auto``, in which this
module will search for the existance of a filesystem matching the
``label``, ``type`` and ``device`` of the ``fs_setup`` entry and
will skip creating the filesystem if one is found.
However, using this "auto" flag always recreates the partition no matter
if it has been done before or not.
This commit fixes a bug in which the "partition" attribute was always
set to None although in some cases it should not.
LP: #1634678
|
|
ef18b8ac4c added support for handing network config v2 (aka netplan
format). This just adds that feature to the list of supported features.
|
|
test_netconfig.py:test_apply_network_config_eni_ub would attempt to
remove any .link files that cloud-init had written. This was just
a failure to mock out all of its interaction with the host.
|
|
Network configuration version 2 format is implemented in a package
called netplan (nplan)[1] which allows consolidated network config
for multiple network controllers.
- Add a new netplan renderer
- Update default policy, placing eni and sysconfig first
This requires explicit policy to enable netplan over eni
on systems which have both (Yakkety, Zesty, UC16)
- Allow any network state (parsed from any format cloud-init supports) to
render to v2 if system supports netplan.
- Move eni's _subnet_is_ipv6 to common code for use by other renderers
- Make sysconfig renderer always emit /etc/syconfig/network configuration
- Update cloud-init.service systemd unit to also wait on
systemd-networkd-wait-online.service
1. https://lists.ubuntu.com/archives/ubuntu-devel/2016-July/039464.html
|
|
Previously, the distro had hard coded which network renderer it would
use. This adds support for just picking the right renderer based
on what is available.
Now, that can be set via a priority in system_info, but should
generally work. That config looks like:
system_info:
network:
renderers: ["eni", "sysconfig"]
When no renderers are found, a specific RendererNotFoundError is raised.
stages.py is modified to catch that and log it at error level. This
path should not really be exercised, but could occur if for example an
Ubuntu system did not have ifupdown, or a rhel system did not have
sysconfig. In such a system previously we would have quietly rendered
ENI configuration but that would have been ignored. This is one step
better in that we at least log the error.
|
|
render_network_state should default to rendering on /.
The changes here just make it so render_network_state does not
require a target, but defaults to None, and uses target_path
to handle that.
|
|
As reported in bug 1671927, sysconfig had an issue with rendering
a loopback device. The problem was that some as yet unknown issue was
causing the openstack config drive to parse the provided ENI file rather
than reading the network_data.json. Parsing an ENI file would add a
a 'lo' device of type 'physical', and sysconfig was failing to render
that.
The change here is:
a.) add a 'loopback' type rather than 'physical' for network config.
{'name': 'lo', 'type': 'loopback', 'subnets': ['type': 'loopback']}
b.) support skipping that type in the eni and sysconfig renderers.
c.) make network_state just piggy back on 'physical' renderer for
loopback (this was what was happening before).
Tests are added for eni and sysconfig renderer.
|
|
Previously, cloud-init would throw an exception if an interface had
both ipv4 and ipv6 addresses and a default gateway for each address
family. This change allows cloud-init to correctly configure
interfaces in this situation.
LP: #1669504
|
|
log a warning rather than raising ValueError if we see more than three
nameserver addresses.
LP: #1670052
|
|
|
|
There were some logic errors in sysconfig.py that appear to be the
result of accidentally typing "iface" where it should have been
"iface_cfg". This patch corrects those problems so that the module
can run successfully.
LP: #1665441
Resolves: rhbz#1389530
|
|
When the ds-identify code landed, it started writing /run/cloud.cfg
but at the moment, nothing was reading that. The result is that
ds-identify only worked to disable cloud-init entirely.
|
|
The iface:alias syntax for eni rendering is brittle with ipv6.
Replace it with using multiple iface stanzas with the same iface
name which is supported. Side-effect is that one can no longer
do 'ifup $iface:$alias' but requires instead use of ip address
{add|delete} instead.
LP: #1657940
|
|
The test_user_data_normalize and test_net files had gotten
the executable bit set, and thus are skipped by nose by default.
We could set run with the --exe flag, but they should not have
gotten this way.
Other changes here:
* replace TempDirTestCase with CiTestCase, which has some nice
tmp_dir() and tmp_path() functions. Going forward the intent
is to have CiTestCase be the base test case for tests.
* test_net: switch to CiTestCase and fix usage that was
silently broken, because of exe bit.
* populate_dir: return the list of files that it writes rather
than having no return value.
* CiTestCase:
* support tmp_path("foo") that returns a full path to 'foo'
under a tmpdir.
* add tmp_dir() to get a temp dir and clean up.
|
|
No shipping ubuntu has a python 3.4 that is less than 3.4.2.
Remove this workaround to avoid unnecessary complexity.
This reverts 04a60cf949.
|
|
cloud-init adds ssh_authorized_keys to the default user and to
root but for root it disables the keys with a prefix command.
However, if the public_key key is of type ecdsa-sha2-nistp521,
it is not parsed correctly, and the prefix command is not prepended.
Resolves: rhbz#1151824
LP: #1658174
|
|
this ensures that we call httpretty.reset() before calling
httppretty.register_uri(...), which ensures that we get a fresh
callback with the expected version of the metadata.
LP: #1658200
|
|
On EC2, instance metadata can include credentials that remain valid for as
much as 6 hours. Reading these and allowing them to be pickled represents
a potential vulnerability if a snapshot of the disk is taken and shared as
part of an AMI.
This skips security-credentials when walking the meta-data tree.
LP: #1638312
Reviewed-by: Ian Weller <iweller@amazon.com>
Reviewed-by: Ben Cressey <bcressey@amazon.com>
Reported-by: Kyle Barnes <barnesky@amazon.com>
|
|
This modifies get_data in DataSourceOpenStack.py to get the timeout
and retries values from the data source configuration, rather than
from keyword arguments. This permits get_data to use the same timeout
as other methods, and allows an operator to increase the timeout in
environments where the metadata service takes longer than five seconds
to respond.
LP: #1657130
Resolves: rhbz#1408589
|
|
Failures to load the kernel command line's url (cloud-config-url=)
would previously get swallowed. This should make it much more
obvious when that happens. With logging going to expected places
at sane levels (WARN will go to stderr by default).
|
|
This has been a recurring ask and we had initially just made the change to
the cloud-init 2.0 codebase. As the current thinking is we'll just
continue to enhance the current codebase, its desirable to relicense to
match what we'd intended as part of the 2.0 plan here.
- put a brief description of license in LICENSE file
- put full license versions in LICENSE-GPLv3 and LICENSE-Apache2.0
- simplify the per-file header to reference LICENSE
- tox: ignore H102 (Apache License Header check)
Add license header to files that ship.
Reformat headers, make sure everything has vi: at end of file.
Non-shipping files do not need the copyright header,
but at the moment tests/ have it.
|
|
The correct order of precedence when reading the base config:
builtin config
system config
kernel command line provided config.
This reverts commit 63501f44, which actually broke the behavior it
reported to fix. It also adds some unit tests to ensure this behavior
is not broken again.
LP: #1582323
|
|
This just adds a unit test for a case found to be failing in curtin.
The issue was reported under bug 1649652.
|
