Add LXD datasource (#1040)

Add DataSourceLXD which knows how to talk to the dev-lxd socket to
obtain all instance metadata API:
https://linuxcontainers.org/lxd/docs/master/dev-lxd.

This first branch is to deliver feature parity with the existing
NoCloud datasource which is currently used to intialize LXC instances
on first boot.

Introduce a SocketConnectionPool and LXDSocketAdapter to support
performing HTTP GETs on the following routes which are surfaced by the
LXD host to all containers:
http://unix.socket/1.0/meta-data
http://unix.socket/1.0/config/user.user-data
http://unix.socket/1.0/config/user.network-config
http://unix.socket/1.0/config/user.vendor-data
These 4 routes minimally replace the static content provided in the
following nocloud-net seed files:
/var/lib/cloud/nocloud-net/{meta-data,vendor-data,user-data,network-config}

The intent of this commit is to set a foundation for LXD socket
communication that will allow us to build network hot-plug features
by eventually consuming LXD's websocket upgrade route 1.0/events to
react to network, meta-data and user-data config changes over time.

In the event that no custom network-config is provided, default to the
same network-config definition provided by LXD to the NoCloud
network-config seed file.

Supplemental features above NoCloud datasource:
surface all custom instance data config keys via cloud-init query ds
which aids in discoverability of features/tags/labels as well as
conditional #cloud-config jinja templates operations based on custom
config options.
TBD: better cloud-init query support for dot-delimited keys

2 files changed, 66 insertions, 0 deletions

diff --git a/doc/rtd/topics/datasources.rst b/doc/rtd/topics/datasources.rst
index f5aee1c2..0ebc0f32 100644
--- a/doc/rtd/topics/datasources.rst
+++ b/doc/rtd/topics/datasources.rst
@@ -39,6 +39,7 @@ The following is a list of documents for each supported datasource:
    datasources/exoscale.rst
    datasources/fallback.rst
    datasources/gce.rst
+   datasources/lxd.rst
    datasources/maas.rst
    datasources/nocloud.rst
    datasources/opennebula.rst
diff --git a/doc/rtd/topics/datasources/lxd.rst b/doc/rtd/topics/datasources/lxd.rst
new file mode 100644
index 00000000..3991a4dd
--- /dev/null
+++ b/doc/rtd/topics/datasources/lxd.rst
@@ -0,0 +1,65 @@
+.. _datasource_lxd:
+
+LXD
+===
+
+The data source ``LXD`` allows the user to provide custom user-data,
+vendor-data, meta-data and network-config to the instance without running
+a network service (or even without having a network at all). This datasource
+performs HTTP GETs against the `LXD socket device`_ which is provided to each
+running LXD container and VM as ``/dev/lxd/sock`` and represents all
+instance-metadata as versioned HTTP routes such as:
+
+  - 1.0/meta-data
+  - 1.0/config/user.meta-data
+  - 1.0/config/user.vendor-data
+  - 1.0/config/user.user-data
+  - 1.0/config/user.<any-custom-key>
+
+The LXD socket device ``/dev/lxd/sock`` is only present on containers and VMs
+when the instance configuration has ``security.devlxd=true`` (default).
+Disabling ``security.devlxd`` configuration setting at initial launch will
+ensure that cloud-init uses the :ref:`datasource_nocloud` datasource.
+Disabling ``security.devlxd`` ove the life of the container will result in
+warnings from cloud-init and cloud-init will keep the originally detected LXD
+datasource.
+
+The LXD datasource provides cloud-init the opportunity to react to meta-data,
+vendor-data, user-data and network-config changes and render the updated
+configuration across a system reboot.
+
+One can manipulate what meta-data, vendor-data or user-data is provided to
+the launched container using the LXD profiles or
+``lxc launch ... -c <key>="<value>"`` at initial container launch using one of
+the following keys:
+
+ - user.meta-data: YAML metadata which will be appended to base meta-data
+ - user.vendor-data: YAML which overrides any meta-data values
+ - user.network-config: YAML representing either :ref:`network_config_v1` or
+   :ref:`network_config_v2` format
+ - user.user-data: YAML which takes preference and overrides both meta-data
+   and vendor-data values
+ - user.any-key: Custom user configuration key and value pairs can be passed to
+   cloud-init. Those keys/values will be present in instance-data which can be
+   used by both `#template: jinja` #cloud-config templates and
+   the `cloud-init query` command.
+
+
+By default, network configuration from this datasource will be:
+
+.. code:: yaml
+
+  version: 1
+  config:
+      - type: physical
+        name: eth0
+        subnets:
+            - type: dhcp
+              control: auto
+
+This datasource is intended to replace :ref:`datasource_nocloud`
+datasource for LXD instances with a more direct support for LXD APIs instead
+of static NoCloud seed files.
+
+.. _LXD socket device: https://linuxcontainers.org/lxd/docs/master/dev-lxd
+.. vi: textwidth=78