Add initial docs about datasources.

Start moving the current README for
datasources to a RST format and include
those files in the rtd site.

LP: #1113650

6 files changed, 281 insertions, 238 deletions

diff --git a/doc/sources/altcloud/README b/doc/sources/altcloud/README
deleted file mode 100644
index 87d7949a..00000000
--- a/doc/sources/altcloud/README
+++ /dev/null
@@ -1,65 +0,0 @@
-Data souce AltCloud will be used to pick up user data on
-RHEVm and vSphere.
-
-RHEVm:
-======
-For REHVm v3.0 the userdata is injected into the VM using floppy
-injection via the RHEVm dashboard "Custom Properties". The format
-of the Custom Properties entry must be:
-"floppyinject=user-data.txt:<base64 encoded data>"
-
-e.g.: To pass a simple bash script
-
-% cat simple_script.bash
-#!/bin/bash
-echo "Hello Joe!" >> /tmp/JJV_Joe_out.txt
-
-% base64 < simple_script.bash
-IyEvYmluL2Jhc2gKZWNobyAiSGVsbG8gSm9lISIgPj4gL3RtcC9KSlZfSm9lX291dC50eHQK
-
-To pass this example script to cloud-init running in a RHEVm v3.0 VM
-set the "Custom Properties" when creating the RHEMv v3.0 VM to:
-floppyinject=user-data.txt:IyEvYmluL2Jhc2gKZWNobyAiSGVsbG8gSm9lISIgPj4gL3RtcC9KSlZfSm9lX291dC50eHQK
-
-NOTE: The prefix with file name must be: "floppyinject=user-data.txt:"
-
-It is also possible to launch a RHEVm v3.0 VM and pass optional user
-data to it using the Delta Cloud.
-For more inforation on Delta Cloud see: http://deltacloud.apache.org
-
-vSphere:
-========
-For VMWare's vSphere the userdata is injected into the VM an ISO
-via the cdrom. This can be done using the vSphere dashboard 
-by connecting an ISO image to the CD/DVD drive.
-
-To pass this example script to cloud-init running in a vSphere VM
-set the CD/DVD drive when creating the vSphere VM to point to an
-ISO on the data store.
-
-The ISO must contain the user data:
-
-For example, to pass the same simple_script.bash to vSphere:
-
-Create the ISO:
-===============
-% mkdir my-iso
-
-NOTE: The file name on the ISO must be: "user-data.txt"
-% cp simple_scirpt.bash my-iso/user-data.txt
-
-% genisoimage -o user-data.iso -r my-iso
-
-Verify the ISO:
-===============
-% sudo mkdir /media/vsphere_iso
-% sudo mount -o loop JoeV_CI_02.iso /media/vsphere_iso
-% cat /media/vsphere_iso/user-data.txt
-% sudo umount /media/vsphere_iso
-
-Then, launch the vSphere VM the ISO user-data.iso attached as a CDrom.
-
-It is also possible to launch a vSphere VM and pass optional user
-data to it using the Delta Cloud.
-
-For more inforation on Delta Cloud see: http://deltacloud.apache.org
diff --git a/doc/sources/altcloud/README.rst b/doc/sources/altcloud/README.rst
new file mode 100644
index 00000000..b5d72ebb
--- /dev/null
+++ b/doc/sources/altcloud/README.rst
@@ -0,0 +1,87 @@
+The datasource altcloud will be used to pick up user data on `RHEVm`_ and `vSphere`_.
+
+RHEVm
+~~~~~~
+
+For `RHEVm`_ v3.0 the userdata is injected into the VM using floppy
+injection via the `RHEVm`_ dashboard "Custom Properties". 
+
+The format of the Custom Properties entry must be:
+
+::
+    
+    floppyinject=user-data.txt:<base64 encoded data>
+
+For example to pass a simple bash script:
+
+::
+    
+    % cat simple_script.bash
+    #!/bin/bash
+    echo "Hello Joe!" >> /tmp/JJV_Joe_out.txt
+
+    % base64 < simple_script.bash
+    IyEvYmluL2Jhc2gKZWNobyAiSGVsbG8gSm9lISIgPj4gL3RtcC9KSlZfSm9lX291dC50eHQK
+
+To pass this example script to cloud-init running in a  `RHEVm`_ v3.0 VM
+set the "Custom Properties" when creating the RHEMv v3.0 VM to:
+
+::
+
+    floppyinject=user-data.txt:IyEvYmluL2Jhc2gKZWNobyAiSGVsbG8gSm9lISIgPj4gL3RtcC9KSlZfSm9lX291dC50eHQK
+
+**NOTE:** The prefix with file name must be: ``floppyinject=user-data.txt:``
+
+It is also possible to launch a `RHEVm`_ v3.0 VM and pass optional user
+data to it using the Delta Cloud. 
+
+For more information on Delta Cloud see: http://deltacloud.apache.org
+
+vSphere
+~~~~~~~~
+
+For VMWare's `vSphere`_ the userdata is injected into the VM as an ISO
+via the cdrom. This can be done using the `vSphere`_ dashboard 
+by connecting an ISO image to the CD/DVD drive.
+
+To pass this example script to cloud-init running in a `vSphere`_ VM
+set the CD/DVD drive when creating the vSphere VM to point to an
+ISO on the data store. 
+
+**Note:** The ISO must contain the user data.
+
+For example, to pass the same ``simple_script.bash`` to vSphere:
+
+Create the ISO
+-----------------
+
+::
+    
+    % mkdir my-iso
+
+NOTE: The file name on the ISO must be: ``user-data.txt``
+
+::
+    
+    % cp simple_scirpt.bash my-iso/user-data.txt
+    % genisoimage -o user-data.iso -r my-iso
+
+Verify the ISO
+-----------------
+
+::
+    
+    % sudo mkdir /media/vsphere_iso
+    % sudo mount -o loop JoeV_CI_02.iso /media/vsphere_iso
+    % cat /media/vsphere_iso/user-data.txt
+    % sudo umount /media/vsphere_iso
+
+Then, launch the `vSphere`_ VM the ISO user-data.iso attached as a CDROM.
+
+It is also possible to launch a `vSphere`_ VM and pass optional user
+data to it using the Delta Cloud. 
+
+For more information on Delta Cloud see: http://deltacloud.apache.org
+
+.. _RHEVm: https://www.redhat.com/virtualization/rhev/desktop/rhevm/
+.. _vSphere: https://www.vmware.com/products/datacenter-virtualization/vsphere/overview.html
diff --git a/doc/sources/configdrive/README b/doc/sources/configdrive/README
deleted file mode 100644
index ed9033c9..00000000
--- a/doc/sources/configdrive/README
+++ /dev/null
@@ -1,118 +0,0 @@
-The 'ConfigDrive' DataSource supports the OpenStack configdrive disk.
-See doc/source/api_ext/ext_config_drive.rst in the nova source code for
-more information on config drive.
-
-The following criteria are required to be identified by
-DataSourceConfigDrive as a config drive:
- * must be formated with vfat filesystem
- * must be a un-partitioned block device (/dev/vdb, not /dev/vdb1)
- * must contain one of the following files:
-    * etc/network/interfaces
-    * root/.ssh/authorized_keys
-    * meta.js
-
-By default, cloud-init does not consider this source to be a full-fledged
-datasource.  Instead, the default behavior is to assume it is really only
-present to provide networking information.  Cloud-init will copy off the
-network information, apply it to the system, and then continue on.  The
-"full" datasource would then be found in the EC2 metadata service.
-
-== Content of config-drive ==
-  * etc/network/interfaces
-    This file is laid down by nova in order to pass static networking
-    information to the guest.  Cloud-init will copy it off of the config-drive
-    and into /etc/network/interfaces as soon as it can, and then attempt to
-    bring up all network interfaces.
-
-  * root/.ssh/authorized_keys
-    This file is laid down by nova, and contains the keys that were
-    provided to it on instance creation (nova-boot --key ....)
-
-    Cloud-init will copy those keys and put them into the configured user
-    ('ubuntu') .ssh/authorized_keys.
-
-  * meta.js
-    meta.js is populated on the config-drive in response to the user passing
-    "meta flags" (nova boot --meta key=value ...).  It is expected to be json
-    formated.
-
-== Configuration ==
-Cloud-init's behavior can be modified by keys found in the meta.js file in
-the following ways:
- * dsmode:  
-   values: local, net, pass
-   default: pass
-
-   This is what indicates if configdrive is a final data source or not.
-   By default it is 'pass', meaning this datasource should not be read.
-   Set it to 'local' or 'net' to stop cloud-init from continuing on to
-   search for other data sources after network config.
-
-   The difference between 'local' and 'net' is that local will not require
-   networking to be up before user-data actions (or boothooks) are run.
-   
- * instance-id:
-   default: iid-dsconfigdrive
-   This is utilized as the metadata's instance-id.  It should generally
-   be unique, as it is what is used to determine "is this a new instance".
-
- * public-keys:
-   default: None
-   if present, these keys will be used as the public keys for the
-   instance.  This value overrides the content in authorized_keys.
-   Note: it is likely preferable to provide keys via user-data
-
- * user-data:
-   default: None
-   This provides cloud-init user-data.  See other documentation for what
-   all can be present here.
-
-== Example ==
-Here is an example using the nova client (python-novaclien)
-
-Assuming the following variables set up:
- * img_id : set to the nova image id (uuid from image-list)
- * flav_id : set to numeric flavor_id (nova flavor-list)
- * keyname : set to name of key for this instance (nova keypair-list)
-
-$ cat my-user-data 
-#!/bin/sh
-echo ==== USER_DATA FROM EC2 MD ==== | tee /ud.log
-
-$ ud_value=$(sed 's,EC2 MD,META KEY,')
-
-## Now, 'ud_value' has same content of my-user-data file, but
-## with the string "USER_DATA FROM META KEY"
-
-## launch an instance with dsmode=pass
-## This will really not use the configdrive for anything as the mode
-## for the datasource is 'pass', meaning it will still expect some
-## other data source (DataSourceEc2).
-
-$ nova boot --image=$img_id --config-drive=1 --flavor=$flav_id \
-   --key_name=$keyname \
-   --user_data=my-user-data \
-   "--meta=instance-id=iid-001 \
-   "--meta=user-data=${ud_keyval}" \
-   "--meta=dsmode=pass" cfgdrive-dsmode-pass
-
-$ euca-get-console-output i-0000001 | grep USER_DATA
-echo ==== USER_DATA FROM EC2 MD ==== | tee /ud.log
-
-## Now, launch an instance with dsmode=local
-## This time, the only metadata and userdata available to cloud-init
-## are on the config-drive
-$ nova boot --image=$img_id --config-drive=1 --flavor=$flav_id \
-   --key_name=$keyname \
-   --user_data=my-user-data \
-   "--meta=instance-id=iid-001 \
-   "--meta=user-data=${ud_keyval}" \
-   "--meta=dsmode=local" cfgdrive-dsmode-local
-
-$ euca-get-console-output i-0000002 | grep USER_DATA
-echo ==== USER_DATA FROM META KEY ==== | tee /ud.log
-
---
-[1] https://github.com/openstack/nova/blob/master/doc/source/api_ext/ext_config_drive.rst for more if
-
-
diff --git a/doc/sources/configdrive/README.rst b/doc/sources/configdrive/README.rst
new file mode 100644
index 00000000..797872ad
--- /dev/null
+++ b/doc/sources/configdrive/README.rst
@@ -0,0 +1,123 @@
+The configuration drive datasource supports the `OpenStack`_ configuration drive disk.
+
+  See `the config drive extension`_ and `introduction`_ in the public
+  documentation for more information.
+
+By default, cloud-init does *always* consider this source to be a full-fledged
+datasource.  Instead, the typical behavior is to assume it is really only
+present to provide networking information.  Cloud-init will copy off the
+network information, apply it to the system, and then continue on.  The
+"full" datasource could then be found in the EC2 metadata service. If this is
+not the case then the files contained on the located drive must provide equivalents
+to what the EC2 metadata service would provide (which is typical of the version
+2 support listed below)
+
+Version 1
+~~~~~~~~~
+
+The following criteria are required to as a config drive:
+
+1. Must be formatted with `vfat`_ filesystem
+2. Must be a un-partitioned block device (/dev/vdb, not /dev/vdb1)
+3. Must contain *one* of the following files
+
+::
+
+  /etc/network/interfaces
+  /root/.ssh/authorized_keys
+  /meta.js
+
+``/etc/network/interfaces``
+
+    This file is laid down by nova in order to pass static networking
+    information to the guest.  Cloud-init will copy it off of the config-drive
+    and into /etc/network/interfaces (or convert it to RH format) as soon as it can,
+    and then attempt to bring up all network interfaces.
+
+``/root/.ssh/authorized_keys``
+
+    This file is laid down by nova, and contains the ssk keys that were
+    provided to nova on instance creation (nova-boot --key ....)
+
+``/meta.js``
+
+    meta.js is populated on the config-drive in response to the user passing
+    "meta flags" (nova boot --meta key=value ...). It is expected to be json
+    formatted.
+
+Version 2
+~~~~~~~~~~~
+
+The following criteria are required to as a config drive:
+
+1. Must be formatted with `vfat`_ or `iso9660`_ filesystem
+   or have a *filesystem* label of **config-2**
+2. Must be a un-partitioned block device (/dev/vdb, not /dev/vdb1)
+3. The files that will typically be present in the config drive are:
+
+::
+
+  openstack/
+    - 2012-08-10/ or latest/ 
+      - meta_data.json
+      - user_data (not mandatory)
+    - content/
+      - 0000 (referenced content files)
+      - 0001
+      - ....
+  ec2
+    - latest/
+      - meta-data.json (not mandatory)
+
+Keys and values
+~~~~~~~~~~~
+
+Cloud-init's behavior can be modified by keys found in the meta.js (version 1 only) file in the following ways.
+
+::
+
+   dsmode:  
+     values: local, net, pass
+     default: pass
+
+
+This is what indicates if configdrive is a final data source or not.
+By default it is 'pass', meaning this datasource should not be read.
+Set it to 'local' or 'net' to stop cloud-init from continuing on to
+search for other data sources after network config.
+
+The difference between 'local' and 'net' is that local will not require
+networking to be up before user-data actions (or boothooks) are run.
+
+::
+    
+   instance-id:
+     default: iid-dsconfigdrive
+     
+This is utilized as the metadata's instance-id.  It should generally
+be unique, as it is what is used to determine "is this a new instance".
+
+::
+
+   public-keys:
+     default: None
+  
+If present, these keys will be used as the public keys for the
+instance.  This value overrides the content in authorized_keys.
+
+Note: it is likely preferable to provide keys via user-data
+
+::
+    
+   user-data:
+     default: None
+     
+This provides cloud-init user-data. See :ref:`examples <yaml_examples>` for 
+what all can be present here.
+
+.. _OpenStack: http://www.openstack.org/
+.. _introduction: http://docs.openstack.org/trunk/openstack-compute/admin/content/config-drive.html
+.. _python-novaclient: https://github.com/openstack/python-novaclient
+.. _iso9660: https://en.wikipedia.org/wiki/ISO_9660
+.. _vfat: https://en.wikipedia.org/wiki/File_Allocation_Table
+.. _the config drive extension: http://docs.openstack.org/developer/nova/api_ext/ext_config_drive.html
diff --git a/doc/sources/nocloud/README b/doc/sources/nocloud/README
deleted file mode 100644
index c94b206a..00000000
--- a/doc/sources/nocloud/README
+++ /dev/null
@@ -1,55 +0,0 @@
-The data source 'NoCloud' and 'NoCloudNet' allow the user to provide user-data
-and meta-data to the instance without running a network service (or even without
-having a network at all)
-
-You can provide meta-data and user-data to a local vm boot via files on a vfat
-or iso9660 filesystem.  These user-data and meta-data files are expected to be
-in the format described in doc/example/seed/README .  Basically, user-data is
-simply user-data and meta-data is a yaml formated file representing what you'd
-find in the EC2 metadata service.
-
-Given a disk 12.04 cloud image in 'disk.img', you can create a sufficient disk
-by following the example below.
-
-## create user-data and meta-data files that will be used
-## to modify image on first boot
-$ { echo instance-id: iid-local01; echo local-hostname: cloudimg; } > meta-data
-
-$ printf "#cloud-config\npassword: passw0rd\nchpasswd: { expire: False }\nssh_pwauth: True\n" > user-data
-
-## create a disk to attach with some user-data and meta-data
-$ genisoimage  -output seed.iso -volid cidata -joliet -rock user-data meta-data
-
-## alternatively, create a vfat filesystem with same files
-## $ truncate --size 2M seed.img
-## $ mkfs.vfat -n cidata seed.img
-## $ mcopy -oi seed.img user-data meta-data ::
-
-## create a new qcow image to boot, backed by your original image
-$ qemu-img create -f qcow2 -b disk.img boot-disk.img
-
-## boot the image and login as 'ubuntu' with password 'passw0rd'
-## note, passw0rd was set as password through the user-data above,
-## there is no password set on these images.
-$ kvm -m 256 \
-   -net nic -net user,hostfwd=tcp::2222-:22 \
-   -drive file=boot-disk.img,if=virtio \
-   -drive file=seed.iso,if=virtio
-
-Note, that the instance-id provided ('iid-local01' above) is what is used to
-determine if this is "first boot".  So if you are making updates to user-data
-you will also have to change that, or start the disk fresh.
-
-
-Also, you can inject an /etc/network/interfaces file by providing the content
-for that file in the 'network-interfaces' field of metadata.  Example metadata:
-  instance-id: iid-abcdefg
-  network-interfaces: |
-    iface eth0 inet static
-    address 192.168.1.10
-    network 192.168.1.0
-    netmask 255.255.255.0
-    broadcast 192.168.1.255
-    gateway 192.168.1.254
-  hostname: myhost
-
diff --git a/doc/sources/nocloud/README.rst b/doc/sources/nocloud/README.rst
new file mode 100644
index 00000000..aa3cf1a3
--- /dev/null
+++ b/doc/sources/nocloud/README.rst
@@ -0,0 +1,71 @@
+The data source ``NoCloud`` and ``NoCloudNet`` allow the user to provide user-data
+and meta-data to the instance without running a network service (or even without
+having a network at all).
+
+You can provide meta-data and user-data to a local vm boot via files on a `vfat`_
+or `iso9660`_ filesystem.  
+
+These user-data and meta-data files are expected to be
+in the following format.
+
+::
+
+  /user-data
+  /meta-data
+
+Basically, user-data is simply user-data and meta-data is a yaml formatted file
+representing what you'd find in the EC2 metadata service.
+
+Given a disk ubuntu 12.04 cloud image in 'disk.img', you can create a sufficient disk
+by following the example below.
+
+::
+    
+    ## create user-data and meta-data files that will be used
+    ## to modify image on first boot
+    $ { echo instance-id: iid-local01; echo local-hostname: cloudimg; } > meta-data
+    
+    $ printf "#cloud-config\npassword: passw0rd\nchpasswd: { expire: False }\nssh_pwauth: True\n" > user-data
+    
+    ## create a disk to attach with some user-data and meta-data
+    $ genisoimage  -output seed.iso -volid cidata -joliet -rock user-data meta-data
+    
+    ## alternatively, create a vfat filesystem with same files
+    ## $ truncate --size 2M seed.img
+    ## $ mkfs.vfat -n cidata seed.img
+    ## $ mcopy -oi seed.img user-data meta-data ::
+    
+    ## create a new qcow image to boot, backed by your original image
+    $ qemu-img create -f qcow2 -b disk.img boot-disk.img
+    
+    ## boot the image and login as 'ubuntu' with password 'passw0rd'
+    ## note, passw0rd was set as password through the user-data above,
+    ## there is no password set on these images.
+    $ kvm -m 256 \
+       -net nic -net user,hostfwd=tcp::2222-:22 \
+       -drive file=boot-disk.img,if=virtio \
+       -drive file=seed.iso,if=virtio
+
+**Note:** that the instance-id provided (``iid-local01`` above) is what is used to
+determine if this is "first boot".  So if you are making updates to user-data
+you will also have to change that, or start the disk fresh.
+
+Also, you can inject an ``/etc/network/interfaces`` file by providing the content
+for that file in the ``network-interfaces`` field of metadata.  
+
+Example metadata:
+
+::
+    
+    instance-id: iid-abcdefg
+    network-interfaces: |
+      iface eth0 inet static
+      address 192.168.1.10
+      network 192.168.1.0
+      netmask 255.255.255.0
+      broadcast 192.168.1.255
+      gateway 192.168.1.254
+    hostname: myhost
+
+.. _iso9660: https://en.wikipedia.org/wiki/ISO_9660
+.. _vfat: https://en.wikipedia.org/wiki/File_Allocation_Table