| Age | Commit message (Collapse) | Author |
|---|
|
With the new `cc_vyos_userdata.py` module is possible to set in User-Data (`#cloud-config`) new parameter `vyos_config_commands`. This parameter should be a list of VyOS configuration commands that will be applied during deployment.
The module will run after the Meta-Data module `cc_vyos.py`.
Commands requirements:
- one command per line
- if command ending by value, it must be inside single quotes: `set some option 'value'`, `delete some option 'value'`
- a single-quote symbol is not allowed inside command or value
The commands list produced by the `show configuration commands` command on a VyOS router should comply with all the requirements, so it is easy to get a proper commands list by copying it from another router.
Usage example (User-Data content):
```
#cloud-config
vyos_config_commands:
- set system host-name 'demo123'
- set system ntp server 1.pool.ntp.org
- set system ntp server 2.pool.ntp.org
- delete interfaces ethernet eth2 address
- set interfaces ethernet eth2 address '192.0.2.1/24'
```
|
|
T3039: Enabled growpart and resizefs (modified) modules
|
|
Since partition size is hardcoded into VyOS images for virtual environments, it is not impossible to use all available space on storage. This change enables two Cloud-init modules:
- `growpart` to grow persistence partition;
- `resizefs_vyos` to resize filesystem size to the whole partition.
The `resizefs_vyos` module forked from the main `resizefs` to allow resizing not only root partition but any custom also.
|
|
cc_vyos: T3028: Added creating of fallback user
|
|
To provide connectivity to the CLI in cases when there were no credentials in metadata added a fallback user `vyos/vyos`.
|
|
cc_vyos: T2117: Revert back to _find_networking_config() function
|
|
In rare cases, when network-config provided not by a true data source, but via kernel argument, initramfs, cloud-config file, the cloud.datasource.network_config variable may be empty. So, to make such configuration available we need to revert back to the previous variant with cloudinit.stages.Init()._find_networking_config().
|
|
cc_vyos: T2117: Revert back to get_hostname_fqdn() function
|
|
Unfortunately, `cloud.get_hostname()` does not work if a hostname was configured using cloud-config. So, we still need to use the `get_hostname_fqdn()` from `cloudinit.util`.
Also, was added configuration for domain-name to handle FQDN properly, if it is available.
|
|
T2117: Cleaned up systemd service for cloud-config
|
|
The startup order control was moved from the `cloud-config.service.tmpl` template to the `vyos-router.service` in the https://github.com/vyos/vyatta-cfg/commit/94f7abdf748c24b70c4741417c74147e52689da3
|
|
cc_vyos: T2117: OVF datasource cleanup
|
|
The `DataSourceOVF.py` file was cleaned up from VyOS-specific changes.
Also was changed related functionality in the `cc_vyos.py`:
- in addition to limited metadata provided by Cloud-init, the function `get_properties` from the `DataSourceOVF.py` used to get unfiltered values from an OVF environment;
- `set_tag` for the `interfaces ethernet` node was moved from multiple places to the `set_ipaddress` function;
- multiple checks for 'null' value in OVF were replaced with the iteration via all values and replacing `null` with `None`. This allows using easier logic during values check;
- simplified conversion of the values from OVF to an IP address;
- added logging for all actions in the `set_config_ovf` function.
|
|
T2117: Cloud-init updated to 20.3
|
|
Merged with 20.3 tag from the upstream Cloud-init repository
|
|
Improved compatibility with upstream version
|
|
VyOS-specific configuration were moved from the `cloud.cfg.tmpl` to the separated file `cloud.cfg.d/10_vyos.cfg`.
With changes in the default template, some build tests failed, and everything from this default configuration can be overwritten in the config file, so there is no strict necessity to keep our changes in the configuration template.
|
|
Since the `instance-data.json` file is the only way offered by Cloud-Init to get formatted metadata, using it instead of `_get_standardized_metadata()` function is safer and must be compatible with all versions.
|
|
cc_vyos: T2726: User creating optimizations and small fixes
|
|
This commit is addressed to solve some old issues with creating users in the system and simplify the parts of the module related to this. Also, some small fixes.
- removed Python modules os, cloudinit.stages, cloudinit.util dependencies. Related functionality replaced by other modules (see below)
- detection of hashed passwords was simplified, made 100% compatible with the rest Cloud-init documentation and recommendations. Also, it was moved from the `handle` function to the `set_pass_login` to reduce the code size and make it more clear
- replaced sequenced SSH public keys enumeration for keys without comments to UUID-based to simplify the code and make the logic easier
- replaced home-growed SSH key parser/checker to the native cloudinit.ssh_util.AuthKeyLineParser()
- added support for SSH key options configuration
- added possibility to use all key types supported by VyOS: 'ssh-dss', 'ssh-rsa', 'ecdsa-sha2-nistp256', 'ecdsa-sha2-nistp384', 'ssh-ed25519', 'ecdsa-sha2-nistp521'
- fixed typo in configuration for `distance`/`metric` option in set_config_interfaces_v1()
- added the stable format of the Meta-Data: `v1`. It must be absolutely equal for any datasource, therefore it is always better to use data from it than from any other sources if this is possible
- added User-Data and Vendor-Data logging. Currently not used for anything, but required for a proper debugging
- replaced datasource source from the unstable metadata field to the stable `cloud.datasource.dsname`
- replaced Network-Config source from `init._find_networking_config()` to the more correct `cloud.datasource.network_config`
- replaced hostname source from the `util.get_hostname_fqdn()` to `cloud.get_hostname()`, what is actually the same, to drop `util` dependency
- the part specific for Azure cloud united with the main part of users creating code, since there is actually no platform-specific functions and everything was moved to the common places, what improved compatibility with the similar environments
- rewritten users creating logic
**Important information about users and credentials**
In the Cloud-init exists multiple ways of how to configure authentication: public keys in Meta-Data, default user name and options in the main config file, several config modules (`cc_set_passwords`, `cc_ssh`, `cc_users_groups`) configurable via `#cloud-config`, maybe something more. Cloud-Init solves this by merging information from most of these sources to a single users' database, but information can overwrite each other.
Very simplified logic description: if something is configured in a User-Data (`#cloud-config`), then most likely default values like username `vyos`, or SSH public keys from Meta-Data will be dropped by Cloud-Init.
This implementation should apply public SSH keys and passwords without associated username to the default user (usually `vyos`, but some platforms may allow using your own). If you are creating any additional user, a default one will not be created and common authentication methods will not be applied, so you need to provide the complete authentication details for it.
|
|
Bump the version in cloudinit/version.py to 20.3 and
update ChangeLog.
LP: #1892878
|
|
Changes:
tox: bump the pylint version to 2.6.0 in the default run
Fix pylint 2.6.0 W0707 warnings (raise-missing-from)
|
|
This fixes a long delay during boot of some instances. For Azure instance types using SR-IOV via the Hyper-V netvsc network driver, two network interfaces are created that share the same MAC, but only the virtual device should be configured and used. Updating the netplan configuration to filter on the hv_netvsc driver prevents netplan from trying to figure both devices.
LP: #1830740
|
|
|
|
* DHCP sandboxing failing on noexec mounted /var/tmp
If /var/tmp is mounted with noexec option the DHCP sandboxing will fail
with Permission Denied. This patch simply avoids this error by checking
the exec permission updating the dhcp path in negative case.
rhbz: https://bugzilla.redhat.com/show_bug.cgi?id=1857309
Signed-off-by: Eduardo Otubo <otubo@redhat.com>
* Replacing with os.* calls
* Adding test and removing isfile() useless call.
Co-authored-by: Rick Harding <rharding@mitechie.com>
|
|
Update ssh_util.py with latest list of keys (from openssh-8.3p1/sshkey.c),
Added keys:
sk-ecdsa-sha2-nistp256-cert-v01@openssh.com
sk-ecdsa-sha2-nistp256@openssh.com
sk-ssh-ed25519-cert-v01@openssh.com
sk-ssh-ed25519@openssh.com
ssh-xmss-cert-v01@openssh.com
ssh-xmss@openssh.com
LP: #1877869
|
|
cloud-init query tries to directly load and decode
raw user-data from /var/lib/cloud/instance/user-data.txt.
This results in UnicodeDecodeErrors on some platforms which
provide compressed content.
Avoid UnicodeDecoderErrors when parsing compressed user-data at
/var/lib/cloud/instance/user-data.txt.
LP: #1889938
|
|
Push the cloud-init.log file (Up to 500KB at once) to the KVP before reporting ready to the Azure platform.
Based on the analysis done on a large sample of cloud-init.log files, Here's the statistics collected on the log file size:
P50 P90 P95 P99 P99.9 P99.99
137K 423K 537K 3.5MB 6MB 16MB
This change limits the size of cloud-init.log file data that gets dumped to KVP to 500KB. So for ~95% of the cases, the whole log file will be dumped and for the remaining ~5%, we will get the last 500KB of the cloud-init.log file.
To asses the performance of the 500KB limit, 250 VM were deployed with a 500KB cloud-init.log file and the time taken to compress, encode and dump the entries to KVP was measured. Here's the time in milliseconds percentiles:
P50 P99 P999
75.705 232.701 1169.636
Another 250 VMs were deployed with this logic dumping their normal cloud-init.log file to KVP, the same timing was measured as above. Here's the time in milliseconds percentiles:
P50 P99 P999
1.88 5.277 6.992
Added excluded_handlers to the report_event function to be able to opt-out from reporting the events of the compressed cloud-init.log file to the cloud-init.log file.
The KVP break_down logic had a bug, where it will reuse the same key for all the split chunks of KVP which results in overwriting the split KVPs by the last one when consumed by Hyper-V. I added the split chunk index as a differentiator to the KVP key.
The Hyper-V consumes the KVPs from the KVP file as chunks whose key is 512KB and value is 2048KB but the Azure platform expects the value to be 1024KB, thus I introduced the Azure value limit.
|
|
Add new module cc_apk_configure for creating Alpine /etc/apk/repositories file.
Modify cc_ca_certs, cc_ntp, cc_power_state_change, and cc_resolv_conf for Alpine.
Add Alpine template files for Chrony and Busybox NTP support.
Add Alpine template file for /etc/hosts.
|
|
According to man page `man 8 swapon', "Preallocated swap files are
supported on XFS since Linux 4.18". This patch checks for kernel version
before attepting to create swapfile, using dd for XFS only on kernel
versions <= 4.18 or btrfs.
Add new func util.kernel_version which returns a tuple of ints (major, minor)
Signed-off-by: Eduardo Otubo otubo@redhat.com
|
|
* cli: add devel make-mime subcommand
Cloud-init documents an in-source-tree tool, make-mime.py used to
help users create multi-part mime user-data. This tool is not shipped
in the cloud-init install and unavailable at runtime. This patch
takes tools/make-mime.py and makes the functionality available via
the devel subcommand.
The primary interface of --attach file:content-type is still present.
The cli now adds:
-l, --list-types Print out a list of supported content-types
-f, --force Ignore errors for unsupported content-types
The tool will now raise a RunTime error if the supplied content-type
is not supported (or more likely a typo:
x-shell-script vs. x-shellscript)
* make-mime: write to stderr and exit 1 instead of raising RuntimeError
* Update example to match docs
* Update docs for make-mime subcommand
* Remove tools/make-mime.py; replaced by cloud-init devel make-mime
Co-authored-by: Rick Harding <rharding@mitechie.com>
|
|
Commit d00126c167fc06d913d99cfc184bf3402cb8cf53 regressed cloud-init
handling in multipart MIME user-data. Specifically, cloud-init would
examine the payload of the MIME part to determine what the content
type and subsequently which handler to use. This meant that user-data
which had shellscript payloads (starts with #!) were always handled
as shellscripts, rather than their declared MIME type and affected
when the payload was handled.
One failing scenario was a MIME part with text/cloud-boothook type
declared and a shellscript payload. This was run at shellscript
processing time rather than boothook time resulting in an change in
behavior from previous cloud-init releases.
To continue to support known scenarios where clouds have specifed
a MIME type of text/x-shellscript but provided a payload of something
other than shellscripts, we're changing the lookup logic to check for
the TYPES_NEEDED (text/plain, text/x-not-multipart) and only
text/x-shellscript.
It is safe to check text/x-shellscript parts as all shellscripts must
include the #! marker and will be detected as text/x-shellscript types.
If the content is missing the #! marker, it will not be excuted. If
the content is detected as something cloud-init supports, such as
#cloud-config the appropriate cloud-init handler will be used.
This change will fix hanldling for parts which were shellscripts but
ran with the wrong handler due to ignoring of the provided mime-type.
LP: #1888822
|
|
|
|
This PR refactors Azure report ready code to include more robust tests and telemetry.
|
|
The version was bumped in c7248059dd2faaaadfbcef5c83e8e8ea166d6767 to
support running on Python 3.7+ systems. Now that we have separate
`xenial` and `xenial-dev` tox environments, we can restore the correct
pinning for `xenial` without breaking `xenial-dev` on developer
machines.
Also drop the `mock` dependency from `xenial-shared-deps`; its removal
was missed in 5f8f85bb38cc972d3d2c705a1ec73db3f690f323.
|
|
* v2 of the API is now default with fallback to v1.
* Refactored the Oracle datasource to fetch version, instance, and vnic metadata simultaneously.
|
|
We have doc8 configured, so we should use it to gate documentation
changes.
|
|
Plus a minor formatting improvement.
|
|
Few of the 'User and Groups' configurations in cloud-config have no effect on
already existing users. This was not documented earlier.
This change set adds that information to documentation.
Signed-off-by: Shreenidhi Shedi <sshedi@vmware.com>
|
|
This aligns their docstrings more closely with their actual behaviour.
|
|
|
|
|
|
The /opc/v1/ metadata endpoints[0] are universally available in Oracle
Cloud Infrastructure and the OpenStack endpoints are considered
deprecated, so we can refactor the data source to use the OPC endpoints
exclusively. This simplifies the datasource code substantially, and
enables use of OPC-specific attributes in future.
[0] https://docs.cloud.oracle.com/en-us/iaas/Content/Compute/Tasks/gettingmetadata.htm
|
|
We missed this one when we updated all the docs for the Canonical team's
management change.
|
|
* Fix a typo in apt pipelining module
Changed `whcih` to `which`.
* Update .github-cla-signers
I have signed the CLA on Canonical's site, adding my username to list of CLA signers.
* Update .github-cla-signers
I need to sort the list alphabetically.
|
|
|
|
Update DataSourceNoCloud and ds-identify to recognize LABEL_FATBOOT labels from blkid.
Also updated associated tests.
LP: #1841466
|
|
Add "sle_hpc" to list of values which are variant 'suse'.
|
|
pytest 6.0.1 fixes the issue we had with pytest 6.0.0.
This reverts commit db5c1c81840638cfe6f08bbd40982b86dd3ecef7.
|
|
cc_vyos: T2403: Resolved IP addresses configuration conflicts
|
