| Age | Commit message (Collapse) | Author |
|---|
|
|
|
cc_vyos: T2117: Revert back to _find_networking_config() function
|
|
The service installed by the CentOS and RHEL 'ntp' package is
ntpd.service
not
ntp.service
Fix that for those two distros.
Also fix chrony service from 'chrony' to 'chronyd'.
LP: #1897915
|
|
|
|
In rare cases, when network-config provided not by a true data source, but via kernel argument, initramfs, cloud-config file, the cloud.datasource.network_config variable may be empty. So, to make such configuration available we need to revert back to the previous variant with cloudinit.stages.Init()._find_networking_config().
|
|
fails (#549)
Azure datasource's `parse_network_config` throws a fatal uncaught exception when an exception is raised during generation of network config from IMDS metadata. This happens when IMDS metadata is invalid/corrupted (such as when it is missing network or interface metadata). This causes the rest of provisioning to fail.
This changes `parse_network_config` to be a non-fatal implementation. Additionally, when generating network config from IMDS metadata fails, fall back on generating fallback network config (`_generate_network_config_from_fallback_config`).
This also changes fallback network config generation (`_generate_network_config_from_fallback_config`) to blacklist an additional driver: `mlx5_core`.
|
|
|
|
|
|
Co-authored-by: Rick Harding <rharding@mitechie.com>
|
|
|
|
|
|
cc_vyos: T2117: Revert back to get_hostname_fqdn() function
|
|
Unfortunately, `cloud.get_hostname()` does not work if a hostname was configured using cloud-config. So, we still need to use the `get_hostname_fqdn()` from `cloudinit.util`.
Also, was added configuration for domain-name to handle FQDN properly, if it is available.
|
|
T2117: Cleaned up systemd service for cloud-config
|
|
The startup order control was moved from the `cloud-config.service.tmpl` template to the `vyos-router.service` in the https://github.com/vyos/vyatta-cfg/commit/94f7abdf748c24b70c4741417c74147e52689da3
|
|
LP: #1888858
|
|
- use `sh` as highlight language of the code block
- change order so that the confusing indentation is less confusing
|
|
cc_vyos: T2117: OVF datasource cleanup
|
|
The `DataSourceOVF.py` file was cleaned up from VyOS-specific changes.
Also was changed related functionality in the `cc_vyos.py`:
- in addition to limited metadata provided by Cloud-init, the function `get_properties` from the `DataSourceOVF.py` used to get unfiltered values from an OVF environment;
- `set_tag` for the `interfaces ethernet` node was moved from multiple places to the `set_ipaddress` function;
- multiple checks for 'null' value in OVF were replaced with the iteration via all values and replacing `null` with `None`. This allows using easier logic during values check;
- simplified conversion of the values from OVF to an IP address;
- added logging for all actions in the `set_config_ovf` function.
|
|
Co-authored-by: Rick Harding <rharding@mitechie.com>
|
|
Under FreeBSD, we want to use "shutdown -p" for poweroff.
Alpine Linux also has some specificities.
We choose to define a method that returns the shutdown command line to
use, rather than a method that actually does the shutdown. This makes it
easier to have the tests in test_handler_power_state do their
verifications.
Two tests are added for the special behaviours that are known so far.
|
|
This was added in d00126c167fc06d913d99cfc184bf3402cb8cf53, but not
removed in ef041fd822a2cf3a4022525e942ce988b1f95180 which removed the
one usage of it from the original commit.
|
|
T2117: Cloud-init updated to 20.3
|
|
Prior to this change, vlans were rendered in sysconfig with
'TYPE=Ethernet', and incorrectly rendered the PHYSDEV based on
the name of the vlan device rather than the 'link' provided
in the network config.
The change here fixes:
* rendering of TYPE=Ethernet for a vlan
* adds a warning if the configured device name is not supported
per the RHEL 7 docs "11.5. Naming Scheme for VLAN Interfaces"
LP: #1788915
LP: #1826608
RHBZ: #1861871
|
|
Merged with 20.3 tag from the upstream Cloud-init repository
|
|
* pull ssh keys from imds first and fall back to ovf if unavailable
* refactor log and diagnostic messages
* refactor the OpenSSLManager instantiation and certificate usage
* fix unit test where exception was being silenced for generate cert
* fix tests now that certificate is not always generated
* add documentation for ssh key retrieval
* add ability to check if http client has security enabled
* refactor certificate logic to GoalState
|
|
Improved compatibility with upstream version
|
|
VyOS-specific configuration were moved from the `cloud.cfg.tmpl` to the separated file `cloud.cfg.d/10_vyos.cfg`.
With changes in the default template, some build tests failed, and everything from this default configuration can be overwritten in the config file, so there is no strict necessity to keep our changes in the configuration template.
|
|
Since the `instance-data.json` file is the only way offered by Cloud-Init to get formatted metadata, using it instead of `_get_standardized_metadata()` function is safer and must be compatible with all versions.
|
|
|
|
Co-authored-by: Rick Harding <rharding@mitechie.com>
|
|
cc_vyos: T2726: User creating optimizations and small fixes
|
|
This commit is addressed to solve some old issues with creating users in the system and simplify the parts of the module related to this. Also, some small fixes.
- removed Python modules os, cloudinit.stages, cloudinit.util dependencies. Related functionality replaced by other modules (see below)
- detection of hashed passwords was simplified, made 100% compatible with the rest Cloud-init documentation and recommendations. Also, it was moved from the `handle` function to the `set_pass_login` to reduce the code size and make it more clear
- replaced sequenced SSH public keys enumeration for keys without comments to UUID-based to simplify the code and make the logic easier
- replaced home-growed SSH key parser/checker to the native cloudinit.ssh_util.AuthKeyLineParser()
- added support for SSH key options configuration
- added possibility to use all key types supported by VyOS: 'ssh-dss', 'ssh-rsa', 'ecdsa-sha2-nistp256', 'ecdsa-sha2-nistp384', 'ssh-ed25519', 'ecdsa-sha2-nistp521'
- fixed typo in configuration for `distance`/`metric` option in set_config_interfaces_v1()
- added the stable format of the Meta-Data: `v1`. It must be absolutely equal for any datasource, therefore it is always better to use data from it than from any other sources if this is possible
- added User-Data and Vendor-Data logging. Currently not used for anything, but required for a proper debugging
- replaced datasource source from the unstable metadata field to the stable `cloud.datasource.dsname`
- replaced Network-Config source from `init._find_networking_config()` to the more correct `cloud.datasource.network_config`
- replaced hostname source from the `util.get_hostname_fqdn()` to `cloud.get_hostname()`, what is actually the same, to drop `util` dependency
- the part specific for Azure cloud united with the main part of users creating code, since there is actually no platform-specific functions and everything was moved to the common places, what improved compatibility with the similar environments
- rewritten users creating logic
**Important information about users and credentials**
In the Cloud-init exists multiple ways of how to configure authentication: public keys in Meta-Data, default user name and options in the main config file, several config modules (`cc_set_passwords`, `cc_ssh`, `cc_users_groups`) configurable via `#cloud-config`, maybe something more. Cloud-Init solves this by merging information from most of these sources to a single users' database, but information can overwrite each other.
Very simplified logic description: if something is configured in a User-Data (`#cloud-config`), then most likely default values like username `vyos`, or SSH public keys from Meta-Data will be dropped by Cloud-Init.
This implementation should apply public SSH keys and passwords without associated username to the default user (usually `vyos`, but some platforms may allow using your own). If you are creating any additional user, a default one will not be created and common authentication methods will not be applied, so you need to provide the complete authentication details for it.
|
|
During the ec2 integration test runs we occasionally see failures in
deleting test instances. Hopefully a newer boto3 will be more robust.
Also bump: paramiko, cryptography, pylxd (now pulling it from pypi).
Tested with a full Xenial EC2 cloud_tests run.
|
|
|
|
This ensures that KVP data can be correctly reported to Azure.
|
|
This reverts commit 8d25d5e6fac39ab3319ec5d37d23196429fb0c95.
|
|
(#546)
Lintian spotted the following issue:
The specified systemd .service file contains both DefaultDependencies=no
and Conflicts=shutdown.target directives without Before=shutdown.target.
This can lead to problems during shutdown because the service may linger
until the very end of shutdown sequence as nothing requests to stop it
before (due to DefaultDependencies=no).
There is race condition between stopping units and systemd getting a
request to exit the main loop, so it may proceed with shutdown before
all pending stop jobs have been processed.
Please add Before=shutdown.target.
[1] https://lintian.debian.org/tags/systemd-service-file-shutdown-problems.html
Co-authored-by: Rick Harding <rharding@mitechie.com>
|
|
* LXD: detach network from profile before deleting it
When cleaning up the bridge network created by default by LXD as part
of the `lxd init` process detach the network its profile before deleting
it. LXD will otherwise refuse to delete it with error:
Error: The network is currently in use.
Discussion with LXD upstream: https://github.com/lxc/lxd/issues/7804.
LP: #1776958
* LXD bridge deletion: fail if bridge exists but can't be deleted
* LXD bridge deletion: remove useless failure logging
|
|
456fb55744a1acc6bd2f464b7656a9c33d0b7ac5 made tools/read-dependencies
and package/brpm distinguish between build dependencies and runtime
dependencies, however packages/redhat/cloud-init.spec.in expects all
the dependencies to be in the 'requires' list, thus missing some build
dependencies.
This change makes cloud-init.spec use 'buildrequires' too.
The build happens to succeed without python3-devel on the epel-8 copr
chroot as it pulls in the epel-rpm-macros package, which in turn depends
on python3-devel. In other words the dependency is satisfied by chance.
Packages building for Python 3 need to explicitly specify BuildRequires:
python3-devel, see: [1].
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/
|
|
|
|
|
|
Bump the version in cloudinit/version.py to 20.3 and
update ChangeLog.
LP: #1892878
|
|
Changes:
tox: bump the pylint version to 2.6.0 in the default run
Fix pylint 2.6.0 W0707 warnings (raise-missing-from)
|
|
This fixes a long delay during boot of some instances. For Azure instance types using SR-IOV via the Hyper-V netvsc network driver, two network interfaces are created that share the same MAC, but only the virtual device should be configured and used. Updating the netplan configuration to filter on the hv_netvsc driver prevents netplan from trying to figure both devices.
LP: #1830740
|
|
|
|
* DHCP sandboxing failing on noexec mounted /var/tmp
If /var/tmp is mounted with noexec option the DHCP sandboxing will fail
with Permission Denied. This patch simply avoids this error by checking
the exec permission updating the dhcp path in negative case.
rhbz: https://bugzilla.redhat.com/show_bug.cgi?id=1857309
Signed-off-by: Eduardo Otubo <otubo@redhat.com>
* Replacing with os.* calls
* Adding test and removing isfile() useless call.
Co-authored-by: Rick Harding <rharding@mitechie.com>
|
|
Update ssh_util.py with latest list of keys (from openssh-8.3p1/sshkey.c),
Added keys:
sk-ecdsa-sha2-nistp256-cert-v01@openssh.com
sk-ecdsa-sha2-nistp256@openssh.com
sk-ssh-ed25519-cert-v01@openssh.com
sk-ssh-ed25519@openssh.com
ssh-xmss-cert-v01@openssh.com
ssh-xmss@openssh.com
LP: #1877869
|
|
cloud-init query tries to directly load and decode
raw user-data from /var/lib/cloud/instance/user-data.txt.
This results in UnicodeDecodeErrors on some platforms which
provide compressed content.
Avoid UnicodeDecoderErrors when parsing compressed user-data at
/var/lib/cloud/instance/user-data.txt.
LP: #1889938
|
|
Push the cloud-init.log file (Up to 500KB at once) to the KVP before reporting ready to the Azure platform.
Based on the analysis done on a large sample of cloud-init.log files, Here's the statistics collected on the log file size:
P50 P90 P95 P99 P99.9 P99.99
137K 423K 537K 3.5MB 6MB 16MB
This change limits the size of cloud-init.log file data that gets dumped to KVP to 500KB. So for ~95% of the cases, the whole log file will be dumped and for the remaining ~5%, we will get the last 500KB of the cloud-init.log file.
To asses the performance of the 500KB limit, 250 VM were deployed with a 500KB cloud-init.log file and the time taken to compress, encode and dump the entries to KVP was measured. Here's the time in milliseconds percentiles:
P50 P99 P999
75.705 232.701 1169.636
Another 250 VMs were deployed with this logic dumping their normal cloud-init.log file to KVP, the same timing was measured as above. Here's the time in milliseconds percentiles:
P50 P99 P999
1.88 5.277 6.992
Added excluded_handlers to the report_event function to be able to opt-out from reporting the events of the compressed cloud-init.log file to the cloud-init.log file.
The KVP break_down logic had a bug, where it will reuse the same key for all the split chunks of KVP which results in overwriting the split KVPs by the last one when consumed by Hyper-V. I added the split chunk index as a differentiator to the KVP key.
The Hyper-V consumes the KVPs from the KVP file as chunks whose key is 512KB and value is 2048KB but the Azure platform expects the value to be 1024KB, thus I introduced the Azure value limit.
|
