doc: Add config info to ec2, openstack and cloudstack datasource docs

Also document instance-data.json on the top-level datasource topic page.

4 files changed, 153 insertions, 8 deletions

diff --git a/doc/rtd/topics/datasources.rst b/doc/rtd/topics/datasources.rst
index 38ba75d8..30e57d85 100644
--- a/doc/rtd/topics/datasources.rst
+++ b/doc/rtd/topics/datasources.rst
@@ -17,6 +17,103 @@ own way) internally a datasource abstract class was created to allow for a
 single way to access the different cloud systems methods to provide this data
 through the typical usage of subclasses.
 
+
+instance-data
+-------------
+For reference, cloud-init stores all the metadata, vendordata and userdata
+provided by a cloud in a json blob at ``/run/cloud-init/instance-data.json``.
+While the json contains datasource-specific keys and names, cloud-init will
+maintain a minimal set of standardized keys that will remain stable on any
+cloud. Standardized instance-data keys will be present under a "v1" key.
+Any datasource metadata cloud-init consumes will all be present under the
+"ds" key.
+
+Below is an instance-data.json example from an OpenStack instance:
+
+.. sourcecode:: json
+
+  {
+   "base64-encoded-keys": [
+    "ds/meta-data/random_seed",
+    "ds/user-data"
+   ],
+   "ds": {
+    "ec2_metadata": {
+     "ami-id": "ami-0000032f",
+     "ami-launch-index": "0",
+     "ami-manifest-path": "FIXME",
+     "block-device-mapping": {
+      "ami": "vda",
+      "ephemeral0": "/dev/vdb",
+      "root": "/dev/vda"
+     },
+     "hostname": "xenial-test.novalocal",
+     "instance-action": "none",
+     "instance-id": "i-0006e030",
+     "instance-type": "m1.small",
+     "local-hostname": "xenial-test.novalocal",
+     "local-ipv4": "10.5.0.6",
+     "placement": {
+      "availability-zone": "None"
+     },
+     "public-hostname": "xenial-test.novalocal",
+     "public-ipv4": "10.245.162.145",
+     "reservation-id": "r-fxm623oa",
+     "security-groups": "default"
+    },
+    "meta-data": {
+     "availability_zone": null,
+     "devices": [],
+     "hostname": "xenial-test.novalocal",
+     "instance-id": "3e39d278-0644-4728-9479-678f9212d8f0",
+     "launch_index": 0,
+     "local-hostname": "xenial-test.novalocal",
+     "name": "xenial-test",
+     "project_id": "e0eb2d2538814...",
+     "random_seed": "A6yPN...",
+     "uuid": "3e39d278-0644-4728-9479-678f92..."
+    },
+    "network_json": {
+     "links": [
+      {
+       "ethernet_mac_address": "fa:16:3e:7d:74:9b",
+       "id": "tap9ca524d5-6e",
+       "mtu": 8958,
+       "type": "ovs",
+       "vif_id": "9ca524d5-6e5a-4809-936a-6901..."
+      }
+     ],
+     "networks": [
+      {
+       "id": "network0",
+       "link": "tap9ca524d5-6e",
+       "network_id": "c6adfc18-9753-42eb-b3ea-18b57e6b837f",
+       "type": "ipv4_dhcp"
+      }
+     ],
+     "services": [
+      {
+       "address": "10.10.160.2",
+       "type": "dns"
+      }
+     ]
+    },
+    "user-data": "I2Nsb3VkLWNvbmZpZ...",
+    "vendor-data": null
+   },
+   "v1": {
+    "availability-zone": null,
+    "cloud-name": "openstack",
+    "instance-id": "3e39d278-0644-4728-9479-678f9212d8f0",
+    "local-hostname": "xenial-test",
+    "region": null
+   }
+  }
+
+
+
+Datasource API
+--------------
 The current interface that a datasource object must provide is the following:
 
 .. sourcecode:: python
diff --git a/doc/rtd/topics/datasources/cloudstack.rst b/doc/rtd/topics/datasources/cloudstack.rst
index 225093a1..a3101ed7 100644
--- a/doc/rtd/topics/datasources/cloudstack.rst
+++ b/doc/rtd/topics/datasources/cloudstack.rst
@@ -4,7 +4,9 @@ CloudStack
 ==========
 
 `Apache CloudStack`_ expose user-data, meta-data, user password and account
-sshkey thru the Virtual-Router. For more details on meta-data and user-data,
+sshkey thru the Virtual-Router. The datasource obtains the VR address via
+dhcp lease information given to the instance.
+For more details on meta-data and user-data,
 refer the `CloudStack Administrator Guide`_. 
 
 URLs to access user-data and meta-data from the Virtual Machine. Here 10.1.1.1
@@ -18,14 +20,26 @@ is the Virtual Router IP:
 
 Configuration
 -------------
+The following configuration can be set for the datasource in system
+configuration (in `/etc/cloud/cloud.cfg` or `/etc/cloud/cloud.cfg.d/`).
 
-Apache CloudStack datasource can be configured as follows:
+The settings that may be configured are:
 
-.. code:: yaml
+ * **max_wait**:  the maximum amount of clock time in seconds that should be
+   spent searching metadata_urls.  A value less than zero will result in only
+   one request being made, to the first in the list. (default: 120)
+ * **timeout**: the timeout value provided to urlopen for each individual http
+   request.  This is used both when selecting a metadata_url and when crawling
+   the metadata service. (default: 50)
 
-    datasource:
-      CloudStack: {}
-      None: {}
+An example configuration with the default values is provided below:
+
+.. sourcecode:: yaml
+
+  datasource:
+   CloudStack:
+    max_wait: 120
+    timeout: 50
     datasource_list:
       - CloudStack
 
diff --git a/doc/rtd/topics/datasources/ec2.rst b/doc/rtd/topics/datasources/ec2.rst
index 3bc66e17..64c325d8 100644
--- a/doc/rtd/topics/datasources/ec2.rst
+++ b/doc/rtd/topics/datasources/ec2.rst
@@ -60,4 +60,34 @@ To see which versions are supported from your cloud provider use the following U
     ...
     latest
 
+
+
+Configuration
+-------------
+The following configuration can be set for the datasource in system
+configuration (in `/etc/cloud/cloud.cfg` or `/etc/cloud/cloud.cfg.d/`).
+
+The settings that may be configured are:
+
+ * **metadata_urls**: This list of urls will be searched for an Ec2
+   metadata service. The first entry that successfully returns a 200 response
+   for <url>/<version>/meta-data/instance-id will be selected.
+   (default: ['http://169.254.169.254', 'http://instance-data:8773']).
+ * **max_wait**:  the maximum amount of clock time in seconds that should be
+   spent searching metadata_urls.  A value less than zero will result in only
+   one request being made, to the first in the list. (default: 120)
+ * **timeout**: the timeout value provided to urlopen for each individual http
+   request.  This is used both when selecting a metadata_url and when crawling
+   the metadata service. (default: 50)
+
+An example configuration with the default values is provided below:
+
+.. sourcecode:: yaml
+
+  datasource:
+   Ec2:
+    metadata_urls: ["http://169.254.169.254:80", "http://instance-data:8773"]
+    max_wait: 120
+    timeout: 50
+
 .. vi: textwidth=78
diff --git a/doc/rtd/topics/datasources/openstack.rst b/doc/rtd/topics/datasources/openstack.rst
index 43592dec..0ea89943 100644
--- a/doc/rtd/topics/datasources/openstack.rst
+++ b/doc/rtd/topics/datasources/openstack.rst
@@ -25,18 +25,22 @@ The settings that may be configured are:
    the metadata service. (default: 10)
  * **retries**: The number of retries that should be done for an http request.
    This value is used only after metadata_url is selected. (default: 5)
+ * **apply_network_config**: A boolean specifying whether to configure the
+   network for the instance based on network_data.json provided by the
+   metadata service. When False, only configure dhcp on the primary nic for
+   this instances. (default: True)
 
-An example configuration with the default values is provided as example below:
+An example configuration with the default values is provided below:
 
 .. sourcecode:: yaml
 
-  #cloud-config
   datasource:
    OpenStack:
     metadata_urls: ["http://169.254.169.254"]
     max_wait: -1
     timeout: 10
     retries: 5
+    apply_network_config: True
 
 
 Vendor Data