diff options
author | Joshua Harlow <harlowja@yahoo-inc.com> | 2013-03-08 15:57:05 -0800 |
---|---|---|
committer | Joshua Harlow <harlowja@yahoo-inc.com> | 2013-03-08 15:57:05 -0800 |
commit | 73cba9d1f841020b0ee1304f204923d994dd5363 (patch) | |
tree | bd3169edfc958b0ecd0eaf91ea244e51963c2ac4 /doc/merging.txt | |
parent | 973747b00af47c94ba0a719452aa823fb688e5ab (diff) | |
download | vyos-cloud-init-73cba9d1f841020b0ee1304f204923d994dd5363.tar.gz vyos-cloud-init-73cba9d1f841020b0ee1304f204923d994dd5363.zip |
Enable the merging.txt to be in .rst format for public viewing
Diffstat (limited to 'doc/merging.txt')
-rw-r--r-- | doc/merging.txt | 179 |
1 files changed, 0 insertions, 179 deletions
diff --git a/doc/merging.txt b/doc/merging.txt deleted file mode 100644 index f719aec8..00000000 --- a/doc/merging.txt +++ /dev/null @@ -1,179 +0,0 @@ -Arriving in 0.7.2 is a new way to handle dictionary merging in cloud-init. ---- - -Overview --------- - -This was done because it has been a common feature request that there be a -way to specify how cloud-config yaml "dictionaries" are merged together when -there are multiple yamls to merge together (say when performing an #include). - -Since previously the merging algorithm was very simple and would only overwrite -and not append lists, or strings, and so on it was decided to create a new and -improved way to merge dictionaries (and there contained objects) together in a -way that is customizable, thus allowing for users who provide cloud-config data -to determine exactly how there objects will be merged. - -For example. - -#cloud-config (1) -run_cmd: - - bash1 - - bash2 - -#cloud-config (2) -run_cmd: - - bash3 - - bash4 - -The previous way of merging the following 2 objects would result in a final -cloud-config object that contains the following. - -#cloud-config (merged) -run_cmd: - - bash3 - - bash4 - -Typically this is not what users want, instead they would likely prefer: - -#cloud-config (merged) -run_cmd: - - bash1 - - bash2 - - bash3 - - bash4 - -This way makes it easier to combine the various cloud-config objects you have -into a more useful list, thus reducing duplication that would have had to -occur in the previous method to accomplish the same result. - -Customizability ---------------- - -Since the above merging algorithm may not always be the desired merging -algorithm (like how the merging algorithm in < 0.7.2 was not always the preferred -one) the concept of customizing how merging can be done was introduced through -a new concept call 'merge classes'. - -A merge class is a class defintion which provides functions that can be used -to merge a given type with another given type. - -An example of one of these merging classes is the following: - -class Merger(object): - def __init__(self, merger, opts): - self._merger = merger - self._overwrite = 'overwrite' in opts - - # This merging algorithm will attempt to merge with - # another dictionary, on encountering any other type of object - # it will not merge with said object, but will instead return - # the original value - # - # On encountering a dictionary, it will create a new dictionary - # composed of the original and the one to merge with, if 'overwrite' - # is enabled then keys that exist in the original will be overwritten - # by keys in the one to merge with (and associated values). Otherwise - # if not in overwrite mode the 2 conflicting keys themselves will - # be merged. - def _on_dict(self, value, merge_with): - if not isinstance(merge_with, (dict)): - return value - merged = dict(value) - for (k, v) in merge_with.items(): - if k in merged: - if not self._overwrite: - merged[k] = self._merger.merge(merged[k], v) - else: - merged[k] = v - else: - merged[k] = v - return merged - -As you can see there is a '_on_dict' method here that will be given a source value -and a value to merge with. The result will be the merged object. This code itself -is called by another merging class which 'directs' the merging to happen by -analyzing the types of the objects to merge and attempting to find a know object -that will merge that type. I will avoid pasting that here, but it can be found -in the mergers/__init__.py file (see LookupMerger and UnknownMerger). - -So following the typical cloud-init way of allowing source code to be downloaded -and used dynamically, it is possible for users to inject there own merging files -to handle specific types of merging as they choose (the basic ones included will -handle lists, dicts, and strings). Note how each merge can have options associated -with it which affect how the merging is performed, for example a dictionary merger -can be told to overwrite instead of attempt to merge, or a string merger can be -told to append strings instead of discarding other strings to merge with. - -How to activate ---------------- - -There are a few ways to activate the merging algorithms, and to customize them -for your own usage. - -1. The first way involves the usage of MIME messages in cloud-init to specify - multipart documents (this is one way in which multiple cloud-config is joined - together into a single cloud-config). Two new headers are looked for, both - of which can define the way merging is done (the first header to exist wins). - These new headers (in lookup order) are 'Merge-Type' and 'X-Merge-Type'. The value - should be a string which will satisfy the new merging format defintion (see - below for this format). -2. The second way is actually specifying the merge-type in the body of the - cloud-config dictionary. There are 2 ways to specify this, either as a string - or as a dictionary (see format below). The keys that are looked up for this - definition are the following (in order), 'merge_how', 'merge_type'. - -*String format* - -The string format that is expected is the following. - -"classname(option1,option2)+classname2(option3,option4)" (and so on) - -The class name there will be connected to class names used when looking for the -class that can be used to merge and options provided will be given to the class -on construction of that class. - -For example, the default string that is used when none is provided is the following: - -"list(extend)+dict()+str(append)" - -*Dictionary format* - -In cases where a dictionary can be used to specify the same information as the -string format (ie option #2 of above) it can be used, for example. - -merge_how: - - name: list - settings: [extend] - - name: dict - settings: [] - - name: str - settings: [append] - -This would be the equivalent format for default string format but in dictionary -form instead of string form. - -Specifying multiple types and its effect ----------------------------------------- - -Now you may be asking yourself, if I specify a merge-type header or dictionary -for every cloud-config that I provide, what exactly happens? - -The answer is that when merging, a stack of 'merging classes' is kept, the -first one on that stack is the default merging classes, this set of mergers -will be used when the first cloud-config is merged with the initial empty -cloud-config dictionary. If the cloud-config that was just merged provided a -set of merging classes (via the above formats) then those merging classes will -be pushed onto the stack. Now if there is a second cloud-config to be merged then -the merging classes from the cloud-config before the first will be used (not the -default) and so on. This way a cloud-config can decide how it will merge with a -cloud-config dictionary coming after it. - -Other uses ----------- - -The default merging algorithm for merging conf.d yaml files (which form a initial -yaml config for cloud-init) was also changed to use this mechanism so its full -benefits (and customization) can also be used there as well. Other places that -used the previous merging are also similar now extensible (metadata merging for -example). |