diff options
Diffstat (limited to 'doc/rtd/topics')
| -rw-r--r-- | doc/rtd/topics/availability.rst | 20 | ||||
| -rw-r--r-- | doc/rtd/topics/capabilities.rst | 24 | ||||
| -rw-r--r-- | doc/rtd/topics/datasources.rst | 192 | ||||
| -rw-r--r-- | doc/rtd/topics/dir_layout.rst | 81 | ||||
| -rw-r--r-- | doc/rtd/topics/examples.rst | 133 | ||||
| -rw-r--r-- | doc/rtd/topics/format.rst | 159 | ||||
| -rw-r--r-- | doc/rtd/topics/hacking.rst | 1 | ||||
| -rw-r--r-- | doc/rtd/topics/modules.rst | 3 | ||||
| -rw-r--r-- | doc/rtd/topics/moreinfo.rst | 12 |
9 files changed, 625 insertions, 0 deletions
diff --git a/doc/rtd/topics/availability.rst b/doc/rtd/topics/availability.rst new file mode 100644 index 00000000..2d58f808 --- /dev/null +++ b/doc/rtd/topics/availability.rst @@ -0,0 +1,20 @@ +============ +Availability +============ + +It is currently installed in the `Ubuntu Cloud Images`_ and also in the official `Ubuntu`_ images available on EC2. + +Versions for other systems can be (or have been) created for the following distributions: + +- Ubuntu +- Fedora +- Debian +- RHEL +- CentOS +- *and more...* + +So ask your distribution provider where you can obtain an image with it built-in if one is not already available ☺ + + +.. _Ubuntu Cloud Images: http://cloud-images.ubuntu.com/ +.. _Ubuntu: http://www.ubuntu.com/ diff --git a/doc/rtd/topics/capabilities.rst b/doc/rtd/topics/capabilities.rst new file mode 100644 index 00000000..63b34270 --- /dev/null +++ b/doc/rtd/topics/capabilities.rst @@ -0,0 +1,24 @@ +===================== +Capabilities +===================== + +- Setting a default locale +- Setting a instance hostname +- Generating instance ssh private keys +- Adding ssh keys to a users ``.ssh/authorized_keys`` so they can log in +- Setting up ephemeral mount points + +User configurability +-------------------- + +`Cloud-init`_ 's behavior can be configured via user-data. + + User-data can be given by the user at instance launch time. + +This is done via the ``--user-data`` or ``--user-data-file`` argument to ec2-run-instances for example. + +* Check your local clients documentation for how to provide a `user-data` string + or `user-data` file for usage by cloud-init on instance creation. + + +.. _Cloud-init: https://launchpad.net/cloud-init diff --git a/doc/rtd/topics/datasources.rst b/doc/rtd/topics/datasources.rst new file mode 100644 index 00000000..59c58805 --- /dev/null +++ b/doc/rtd/topics/datasources.rst @@ -0,0 +1,192 @@ +.. _datasources: + +========= +Datasources +========= +---------- + What is a datasource? +---------- + +Datasources are sources of configuration data for cloud-init that typically come +from the user (aka userdata) or come from the stack that created the configuration +drive (aka metadata). Typical userdata would include files, yaml, and shell scripts +while typical metadata would include server name, instance id, display name and other +cloud specific details. Since there are multiple ways to provide this data (each cloud +solution seems to prefer its own way) internally a datasource abstract class was +created to allow for a single way to access the different cloud systems methods +to provide this data through the typical usage of subclasses. + +The current interface that a datasource object must provide is the following: + +.. sourcecode:: python + + # returns a mime multipart message that contains + # all the various fully-expanded components that + # were found from processing the raw userdata string + # - when filtering only the mime messages targeting + # this instance id will be returned (or messages with + # no instance id) + def get_userdata(self, apply_filter=False) + + # returns the raw userdata string (or none) + def get_userdata_raw(self) + + # returns a integer (or none) which can be used to identify + # this instance in a group of instances which are typically + # created from a single command, thus allowing programatic + # filtering on this launch index (or other selective actions) + @property + def launch_index(self) + + # the data sources' config_obj is a cloud-config formated + # object that came to it from ways other than cloud-config + # because cloud-config content would be handled elsewhere + def get_config_obj(self) + + #returns a list of public ssh keys + def get_public_ssh_keys(self) + + # translates a device 'short' name into the actual physical device + # fully qualified name (or none if said physical device is not attached + # or does not exist) + def device_name_to_device(self, name) + + # gets the locale string this instance should be applying + # which typically used to adjust the instances locale settings files + def get_locale(self) + + @property + def availability_zone(self) + + # gets the instance id that was assigned to this instance by the + # cloud provider or when said instance id does not exist in the backing + # metadata this will return 'iid-datasource' + def get_instance_id(self) + + # gets the fully qualified domain name that this host should be using + # when configuring network or hostname releated settings, typically + # assigned either by the cloud provider or the user creating the vm + def get_hostname(self, fqdn=False) + + def get_package_mirror_info(self) + +--------------------------- +EC2 +--------------------------- + +The EC2 datasource is the oldest and most widely used datasource that cloud-init +supports. This datasource interacts with a *magic* ip that is provided to the +instance by the cloud provider. Typically this ip is ``169.254.169.254`` of which +at this ip a http server is provided to the instance so that the instance can make +calls to get instance userdata and instance metadata. + +Metadata is accessible via the following URL: + +:: + + GET http://169.254.169.254/2009-04-04/meta-data/ + ami-id + ami-launch-index + ami-manifest-path + block-device-mapping/ + hostname + instance-id + instance-type + local-hostname + local-ipv4 + placement/ + public-hostname + public-ipv4 + public-keys/ + reservation-id + security-groups + +Userdata is accessible via the following URL: + +:: + + GET http://169.254.169.254/2009-04-04/user-data + 1234,fred,reboot,true | 4512,jimbo, | 173,,, + +Note that there are multiple versions of this data provided, cloud-init +by default uses **2009-04-04** but newer versions can be supported with +relative ease (newer versions have more data exposed, while maintaining +backward compatibility with the previous versions). + +To see which versions are supported from your cloud provider use the following URL: + +:: + + GET http://169.254.169.254/ + 1.0 + 2007-01-19 + 2007-03-01 + 2007-08-29 + 2007-10-10 + 2007-12-15 + 2008-02-01 + 2008-09-01 + 2009-04-04 + ... + latest + +**Note:** internally in cloudinit the `boto`_ library used to fetch the instance +userdata and instance metadata, feel free to check that library out, it provides +many other useful EC2 functionality. + +--------------------------- +Config Drive +--------------------------- + +.. include:: ../../sources/configdrive/README.rst + +--------------------------- +Alt cloud +--------------------------- + +.. include:: ../../sources/altcloud/README.rst + +--------------------------- +No cloud +--------------------------- + +.. include:: ../../sources/nocloud/README.rst + +--------------------------- +MAAS +--------------------------- + +*TODO* + +For now see: http://maas.ubuntu.com/ + +--------------------------- +CloudStack +--------------------------- + +*TODO* + +--------------------------- +OVF +--------------------------- + +*TODO* + +For now see: https://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/files/head:/doc/sources/ovf/ + +--------------------------- +Fallback/None +--------------------------- + +This is the fallback datasource when no other datasource can be selected. It is +the equivalent of a *empty* datasource in that it provides a empty string as userdata +and a empty dictionary as metadata. It is useful for testing as well as for when +you do not have a need to have an actual datasource to meet your instance +requirements (ie you just want to run modules that are not concerned with any +external data). It is typically put at the end of the datasource search list +so that if all other datasources are not matched, then this one will be so that +the user is not left with an inaccessible instance. + +**Note:** the instance id that this datasource provides is ``iid-datasource-none``. + +.. _boto: http://docs.pythonboto.org/en/latest/ diff --git a/doc/rtd/topics/dir_layout.rst b/doc/rtd/topics/dir_layout.rst new file mode 100644 index 00000000..8815d33d --- /dev/null +++ b/doc/rtd/topics/dir_layout.rst @@ -0,0 +1,81 @@ +========= +Directory layout +========= + +Cloudinits's directory structure is somewhat different from a regular application:: + + /var/lib/cloud/ + - data/ + - instance-id + - previous-instance-id + - datasource + - previous-datasource + - previous-hostname + - handlers/ + - instance + - instances/ + i-00000XYZ/ + - boot-finished + - cloud-config.txt + - datasource + - handlers/ + - obj.pkl + - scripts/ + - sem/ + - user-data.txt + - user-data.txt.i + - scripts/ + - per-boot/ + - per-instance/ + - per-once/ + - seed/ + - sem/ + +``/var/lib/cloud`` + + The main directory containing the cloud-init specific subdirectories. + It is typically located at ``/var/lib`` but there are certain configuration + scenarios where this can be altered. + + TBD, describe this overriding more. + +``data/`` + + Contains information releated to instance ids, datasources and hostnames of the previous + and current instance if they are different. These can be examined as needed to + determine any information releated to a previous boot (if applicable). + +``handlers/`` + + Custom ``part-handlers`` code is written out here. Files that end up here are written + out with in the scheme of ``part-handler-XYZ`` where ``XYZ`` is the handler number (the + first handler found starts at 0). + + +``instance`` + + A symlink to the current ``instances/`` subdirectory that points to the currently + active instance (which is active is dependent on the datasource loaded). + +``instances/`` + + All instances that were created using this image end up with instance identifer + subdirectories (and corresponding data for each instance). The currently active + instance will be symlinked the the ``instance`` symlink file defined previously. + +``scripts/`` + + Scripts that are downloaded/created by the corresponding ``part-handler`` will end up + in one of these subdirectories. + +``seed/`` + + TBD + +``sem/`` + + Cloud-init has a concept of a module sempahore, which basically consists + of the module name and its frequency. These files are used to ensure a module + is only ran `per-once`, `per-instance`, `per-always`. This folder contains + sempaphore `files` which are only supposed to run `per-once` (not tied to the instance id). + diff --git a/doc/rtd/topics/examples.rst b/doc/rtd/topics/examples.rst new file mode 100644 index 00000000..36508bde --- /dev/null +++ b/doc/rtd/topics/examples.rst @@ -0,0 +1,133 @@ +.. _yaml_examples: + +========= +Cloud config examples +========= + +Including users and groups +--------------------------- + +.. literalinclude:: ../../examples/cloud-config-user-groups.txt + :language: yaml + :linenos: + + +Writing out arbitrary files +--------------------------- + +.. literalinclude:: ../../examples/cloud-config-write-files.txt + :language: yaml + :linenos: + + +Adding a yum repository +--------------------------- + +.. literalinclude:: ../../examples/cloud-config-yum-repo.txt + :language: yaml + :linenos: + +Configure an instances trusted CA certificates +------------------------------------------------------ + +.. literalinclude:: ../../examples/cloud-config-ca-certs.txt + :language: yaml + :linenos: + +Configure an instances resolv.conf +------------------------------------------------------ + +*Note:* when using a config drive and a RHEL like system resolv.conf +will also be managed 'automatically' due to the available information +provided for dns servers in the config drive network format. For those +that wish to have different settings use this module. + +.. literalinclude:: ../../examples/cloud-config-resolv-conf.txt + :language: yaml + :linenos: + +Install and run `chef`_ recipes +------------------------------------------------------ + +.. literalinclude:: ../../examples/cloud-config-chef.txt + :language: yaml + :linenos: + +Setup and run `puppet`_ +------------------------------------------------------ + +.. literalinclude:: ../../examples/cloud-config-puppet.txt + :language: yaml + :linenos: + +Add apt repositories +--------------------------- + +.. literalinclude:: ../../examples/cloud-config-add-apt-repos.txt + :language: yaml + :linenos: + +Run commands on first boot +--------------------------- + +.. literalinclude:: ../../examples/cloud-config-boot-cmds.txt + :language: yaml + :linenos: + +.. literalinclude:: ../../examples/cloud-config-run-cmds.txt + :language: yaml + :linenos: + + +Alter the completion message +--------------------------- + +.. literalinclude:: ../../examples/cloud-config-final-message.txt + :language: yaml + :linenos: + +Install arbitrary packages +--------------------------- + +.. literalinclude:: ../../examples/cloud-config-install-packages.txt + :language: yaml + :linenos: + +Run apt or yum upgrade +--------------------------- + +.. literalinclude:: ../../examples/cloud-config-update-packages.txt + :language: yaml + :linenos: + +Adjust mount points mounted +--------------------------- + +.. literalinclude:: ../../examples/cloud-config-mount-points.txt + :language: yaml + :linenos: + +Call a url when finished +--------------------------- + +.. literalinclude:: ../../examples/cloud-config-phone-home.txt + :language: yaml + :linenos: + +Reboot/poweroff when finished +--------------------------- + +.. literalinclude:: ../../examples/cloud-config-power-state.txt + :language: yaml + :linenos: + +Configure instances ssh-keys +--------------------------- + +.. literalinclude:: ../../examples/cloud-config-ssh-keys.txt + :language: yaml + :linenos: + + +.. _chef: http://www.opscode.com/chef/ +.. _puppet: http://puppetlabs.com/ diff --git a/doc/rtd/topics/format.rst b/doc/rtd/topics/format.rst new file mode 100644 index 00000000..eba9533f --- /dev/null +++ b/doc/rtd/topics/format.rst @@ -0,0 +1,159 @@ +========= +Formats +========= + +User data that will be acted upon by cloud-init must be in one of the following types. + +Gzip Compressed Content +------------------------ + +Content found to be gzip compressed will be uncompressed. +The uncompressed data will then be used as if it were not compressed. +This is typically is useful because user-data is limited to ~16384 [#]_ bytes. + +Mime Multi Part Archive +------------------------ + +This list of rules is applied to each part of this multi-part file. +Using a mime-multi part file, the user can specify more than one type of data. + +For example, both a user data script and a cloud-config type could be specified. + +Supported content-types: + +- text/x-include-once-url +- text/x-include-url +- text/cloud-config-archive +- text/upstart-job +- text/cloud-config +- text/part-handler +- text/x-shellscript +- text/cloud-boothook + +Helper script to generate mime messages +~~~~~~~~~~~~~~~~ + +.. code-block:: python + + #!/usr/bin/python + + import sys + + from email.mime.multipart import MIMEMultipart + from email.mime.text import MIMEText + + if len(sys.argv) == 1: + print("%s input-file:type ..." % (sys.argv[0])) + sys.exit(1) + + combined_message = MIMEMultipart() + for i in sys.argv[1:]: + (filename, format_type) = i.split(":", 1) + with open(filename) as fh: + contents = fh.read() + sub_message = MIMEText(contents, format_type, sys.getdefaultencoding()) + sub_message.add_header('Content-Disposition', 'attachment; filename="%s"' % (filename)) + combined_message.attach(sub_message) + + print(combined_message) + + +User-Data Script +------------------------ + +Typically used by those who just want to execute a shell script. + +Begins with: ``#!`` or ``Content-Type: text/x-shellscript`` when using a MIME archive. + +Example +~~~~~~~ + +:: + + $ cat myscript.sh + + #!/bin/sh + echo "Hello World. The time is now $(date -R)!" | tee /root/output.txt + + $ euca-run-instances --key mykey --user-data-file myscript.sh ami-a07d95c9 + +Include File +------------ + +This content is a ``include`` file. + +The file contains a list of urls, one per line. +Each of the URLs will be read, and their content will be passed through this same set of rules. +Ie, the content read from the URL can be gzipped, mime-multi-part, or plain text. + +Begins with: ``#include`` or ``Content-Type: text/x-include-url`` when using a MIME archive. + +Cloud Config Data +----------------- + +Cloud-config is the simplest way to accomplish some things +via user-data. Using cloud-config syntax, the user can specify certain things in a human friendly format. + +These things include: + +- apt upgrade should be run on first boot +- a different apt mirror should be used +- additional apt sources should be added +- certain ssh keys should be imported +- *and many more...* + +**Note:** The file must be valid yaml syntax. + +See the :ref:`yaml_examples` section for a commented set of examples of supported cloud config formats. + +Begins with: ``#cloud-config`` or ``Content-Type: text/cloud-config`` when using a MIME archive. + +Upstart Job +----------- + +Content is placed into a file in ``/etc/init``, and will be consumed by upstart as any other upstart job. + +Begins with: ``#upstart-job`` or ``Content-Type: text/upstart-job`` when using a MIME archive. + +Cloud Boothook +-------------- + +This content is ``boothook`` data. It is stored in a file under ``/var/lib/cloud`` and then executed immediately. +This is the earliest ``hook`` available. Note, that there is no mechanism provided for running only once. The boothook must take care of this itself. +It is provided with the instance id in the environment variable ``INSTANCE_I``. This could be made use of to provide a 'once-per-instance' type of functionality. + +Begins with: ``#cloud-boothook`` or ``Content-Type: text/cloud-boothook`` when using a MIME archive. + +Part Handler +------------ + +This is a ``part-handler``. It will be written to a file in ``/var/lib/cloud/data`` based on its filename (which is generated). +This must be python code that contains a ``list_types`` method and a ``handle_type`` method. +Once the section is read the ``list_types`` method will be called. It must return a list of mime-types that this part-handler handles. + +The ``handle_type`` method must be like: + +.. code-block:: python + + def handle_part(data, ctype, filename, payload): + # data = the cloudinit object + # ctype = "__begin__", "__end__", or the mime-type of the part that is being handled. + # filename = the filename of the part (or a generated filename if none is present in mime data) + # payload = the parts' content + +Cloud-init will then call the ``handle_type`` method once at begin, once per part received, and once at end. +The ``begin`` and ``end`` calls are to allow the part handler to do initialization or teardown. + +Begins with: ``#part-handler`` or ``Content-Type: text/part-handler`` when using a MIME archive. + +Example +~~~~~~~ + +.. literalinclude:: ../../examples/part-handler.txt + :language: python + :linenos: + +Also this `blog`_ post offers another example for more advanced usage. + +.. [#] See your cloud provider for applicable user-data size limitations... +.. _blog: http://foss-boss.blogspot.com/2011/01/advanced-cloud-init-custom-handlers.html diff --git a/doc/rtd/topics/hacking.rst b/doc/rtd/topics/hacking.rst new file mode 100644 index 00000000..96ab88ef --- /dev/null +++ b/doc/rtd/topics/hacking.rst @@ -0,0 +1 @@ +.. include:: ../../../HACKING.rst diff --git a/doc/rtd/topics/modules.rst b/doc/rtd/topics/modules.rst new file mode 100644 index 00000000..d4dd55df --- /dev/null +++ b/doc/rtd/topics/modules.rst @@ -0,0 +1,3 @@ +========= +Modules +========= diff --git a/doc/rtd/topics/moreinfo.rst b/doc/rtd/topics/moreinfo.rst new file mode 100644 index 00000000..19e96af0 --- /dev/null +++ b/doc/rtd/topics/moreinfo.rst @@ -0,0 +1,12 @@ +========= +More information +========= + +Useful external references +------------------------- + +- `The beauty of cloudinit`_ +- `Introduction to cloud-init`_ (video) + +.. _Introduction to cloud-init: http://www.youtube.com/watch?v=-zL3BdbKyGY +.. _The beauty of cloudinit: http://brandon.fuller.name/archives/2011/05/02/06.40.57/ |