rework of DataSource loading.

The DataSources that are loaded are now controlled entirely via
configuration file of 'datasource_list', like:
   datasource_list: [ "NoCloud", "OVF", "Ec2" ]

Each item in that list is a "DataSourceCollection".  for each item
in the list, cloudinit will attempt to load:
  cloudinit.DataSource<item>
and, failing that,
  DataSource<item>

The module is required to have a method named 'get_datasource_list'
in it that takes a single list of "dependencies" and returns
a list of python classes inside the collection that can run needing
only those dependencies.

The dependencies are defines in DataSource.py. Currently:
  DEP_FILESYSTEM = "FILESYSTEM"
  DEP_NETWORK = "NETWORK"

When 'get_datasource_list' is called for the DataSourceOVF module with
[DEP_FILESYSTEM], then DataSourceOVF returns a single item list with a
reference to the 'DataSourceOVF' class.

When 'get_datasource_list' is called for the DataSourceOVF module with
[DEP_FILESYSTEM, DEP_NETWORK], it will return a single item list
with a reference to 'DataSourceOVFNet'.

cloudinit will then instanciate the class and call its 'get_data' method.
if the get_data method returns 'True', then it selects this class as the
selected Datasource.

1 files changed, 53 insertions, 3 deletions

diff --git a/cloudinit/DataSource.py b/cloudinit/DataSource.py
index 316eb5ae..21404ecc 100644
--- a/cloudinit/DataSource.py
+++ b/cloudinit/DataSource.py
@@ -16,16 +16,23 @@
 #    You should have received a copy of the GNU General Public License
 #    along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-import cloudinit
+
+DEP_FILESYSTEM = "FILESYSTEM"
+DEP_NETWORK = "NETWORK"
+
 import UserDataHandler as ud
 
 class DataSource:
     userdata = None
     metadata = None
     userdata_raw = None
+    log = None
 
-    def __init__(self):
-       pass
+    def __init__(self, log=None):
+        if not log:
+            import logging
+            log = logging.log
+        self.log = log
 
     def get_userdata(self):
         if self.userdata == None:
@@ -91,3 +98,46 @@ class DataSource:
                     return("ip-%s" % '-'.join(r))
             except: pass
         return toks[0]
+
+# return a list of classes that have the same depends as 'depends'
+# iterate through cfg_list, loading "DataSourceCollections" modules
+# and calling their "get_datasource_list".
+# return an ordered list of classes that match
+#
+# - modules must be named "DataSource<item>", where 'item' is an entry
+#   in cfg_list
+# - if pkglist is given, it will iterate try loading from that package
+#   ie, pkglist=[ "foo", "" ]
+#     will first try to load foo.DataSource<item>
+#     then DataSource<item>
+def list_sources(cfg_list, depends, pkglist=[]):
+    retlist = []
+    for ds_coll in cfg_list:
+        for pkg in pkglist:
+            if pkg: pkg="%s." % pkg
+            try:
+                mod = __import__("%sDataSource%s" % (pkg, ds_coll))
+                if pkg:
+                    mod = getattr(mod, "DataSource%s" % ds_coll)
+                lister = getattr(mod, "get_datasource_list")
+                retlist.extend(lister(depends))
+                break
+            except:
+                raise
+    return(retlist)
+
+# depends is a list of dependencies (DEP_FILESYSTEM)
+# dslist is a list of 2 item lists
+# dslist = [ 
+#   ( class, ( depends-that-this-class-needs ) )
+# }
+# it returns a list of 'class' that matched these deps exactly
+# it is a helper function for DataSourceCollections
+def list_from_depends(depends, dslist):
+    retlist = [ ]
+    depset = set(depends)
+    for elem in dslist:
+        (cls, deps) = elem
+        if depset == set(deps):
+            retlist.append(cls)
+    return(retlist)