| Age | Commit message (Collapse) | Author |
|---|
|
Applied Black and isort, fixed any linting issues, updated tox.ini
and CI.
|
|
Improve schema validation.
This adds strict validation of config module definitions at testing
time, with plumbing included for future runtime validation. This
eliminates a class of bugs resulting from schemas that have definitions
that are incorrect, but get interpreted by jsonschema as
"additionalProperties" that are therefore ignored.
- Add strict meta-schema for jsonschema unit test validation
- Separate schema from module metadata structure
- Improve type annotations for various functions and data types
Cleanup:
- Remove unused jsonschema "required" elements
- Eliminate manual memoization in schema.py:get_schema(),
reference module.__doc__ directly
|
|
The main idea is to introduce a second module that takes care of
writing files, but in the 'final' stage.
While the introduction of a second module would allow for choosing
the appropriate place withing the order of modules (and stages),
there is no addition top-level directive being added to the cloud
configuration schema. Instead, 'write-files' schema is being extended
to include a 'defer' attribute used only by the 'write-deffered-files'
modules.
The new module 'write-deferred-files' reuses as much as
possible of the 'write-files' functionality.
|
|
Create a schema object for the `apt_configure` module and
validate this schema in the `handle` function of the module.
There are some considerations regarding this PR:
* The `primary` and `security` keys have the exact same properties. I
tried to eliminate this redundancy by moving their properties to a
common place and then just referencing it for both security and
primary. Similar to what is documented here:
https://json-schema.org/understanding-json-schema/structuring.html
under the `Reuse` paragraph. However, this approach does not work,
because the `#` pointer goes to the beginning of the file, which is
a python module instead of a json file, not allowing the pointer to
find the correct definition. What I did was to create a separate dict
for the mirror config and reuse it for primary and security, but
maybe there are better approaches to do that.
* There was no documentation for the config `debconf_selections`. I
tried to infer what it supposed to do by looking at the code and the
`debconf-set-selections` manpage, but my description may not be
accurate or complete.
* Add a _parse_description function to schema.py to render multi-line
preformatted content instead of squashing all whitespace
LP: #1858884
|
|
Clarify in documentation that write_files will create parent folders
for paths that do not already exist.
This obfuscates what the problem is when people erroneously create
files in /tmp despite the warnings in the documentation not to do so.
People naturally assume that their file is absent because the parent folder
did not exist for it to be created in, causing them to add a runcmd block
to create the folder, even though execution order means that this will not
occur until after write_files have all finished.
|
|
Add schema definition to cc_write_files.py
Cloud-config containing write_files config directives will now
emit warnings for invalid config keys or values for the write_files
module.
Add an extension to JSON schema's draft4validator to permit either
binary or text values for 'string' objects.
This allows for JSON schema validating the YAML declaration of binary
valiues in cloud-config using YAML's '!!binary' syntax.
Add the ability to pass a specific module name to
`cloud-init devel schema --docs <module_name>|all` to optionally
limit doc output during development to a single schema doc.
|
|
|
|
Add 'append: true' to write_files entries to append 'content' to file
specified by 'path' key. This modifies the file open mode to append.
|
|
LP: #1727876
|
|
standargs -> standards.
|
|
Instead of passing around a 'log' reference to functions, just import
logging and use that. This is the pattern that is now more common in
cloud-init.
|
|
Unix file modes are usually represented as octal, but they were being
interpreted as decimal, for example 0o644 would be printed as '420'.
Reviewed-by: Tom Kirchner <tjk@amazon.com>
|
|
This has been a recurring ask and we had initially just made the change to
the cloud-init 2.0 codebase. As the current thinking is we'll just
continue to enhance the current codebase, its desirable to relicense to
match what we'd intended as part of the 2.0 plan here.
- put a brief description of license in LICENSE file
- put full license versions in LICENSE-GPLv3 and LICENSE-Apache2.0
- simplify the per-file header to reference LICENSE
- tox: ignore H102 (Apache License Header check)
Add license header to files that ship.
Reformat headers, make sure everything has vi: at end of file.
Non-shipping files do not need the copyright header,
but at the moment tests/ have it.
|
|
This adds lots of config module documentation in a standard format.
It will greatly improve the content at readthedocs.
Additionally:
* Add a 'doc' env to tox.ini
* Changed default highlight language for sphinx conf from python to yaml
most examples in documentation are yaml configs
* Updated datasource examples to highlight sh code properly
|
|
if no permissions were given in a write_files stanza, then
a warning would be emitted.
The fix here is just to special case handling of None.
|
|
When provided with gzipped data, an exception would be raised
because of a conversion to string.
This fixes the issue and adds a test for write_files.
LP: #1565638
|
|
|
|
make pep8 now is silent on precise's pep8 ( 0.6.1-2ubuntu2).
|
|
This implements file writing via cloud-config. It also
* adjusts other code to have user/group parsing in util instead
of in stages.py,
* renames decomp_str to decomp_gzip since it is more meaningful when named
that (as thats all it can decompress).
LP: #1012854
|
|
Adjust the examples file to reflect this.
|
|
|
|
happens first, which will examine the incoming encoding, and decide the
neccasary decoding types needed to get the final resultant string and
then use these normalized decoding types to actually do the final decode.
Also change the name of the config key that is looked up to 'write_files'
since 'files' is pretty generic and could have clashes with other modules.
Add an example that shows how to use this in the different encoding formats
that are supported.
|
|
other code to have user/group parsing in util instead
of in stages.py, renames decomp_str to decomp_gzip since
it is more meaningful when named that (as thats all it can
decompress).
|
