Integration Testing: tox env, pyxld 2.2.3, and revamp framework

Massive update to clean up and greatly enhance the integration testing
framework developed by Wesley Wiedenmeier.

 - Updated tox environment to run integration test 'citest' to utilize
   pylxd 2.2.3
 - Add support for distro feature flags
   - add framework for feature flags to release config with feature groups
     and overrides allowed in any release conf override level
   - add support for feature flags in platform and config handling
   - during collect, skip testcases that require features not supported by
     the image with a warning message
 - Enable additional distros (i.e. centos, debian)
 - Add 'bddeb' command to build a deb from the current working tree
   cleanly in a container, so deps do not have to be installed on host
 - Adds a command line option '--preserve-data' that ensures that
   collected data will be left after tests run. This also allows the
   directory to store collected data in during the run command to be
   specified using '--data-dir'.
 - Updated Read the Docs testing page and doc strings for pep 257
   compliance

1 files changed, 75 insertions, 0 deletions

diff --git a/tests/cloud_tests/run_funcs.py b/tests/cloud_tests/run_funcs.py
new file mode 100644
index 00000000..8ae91120
--- /dev/null
+++ b/tests/cloud_tests/run_funcs.py
@@ -0,0 +1,75 @@
+# This file is part of cloud-init. See LICENSE file for license information.
+
+"""Run functions."""
+
+import os
+
+from tests.cloud_tests import bddeb, collect, util, verify
+
+
+def tree_collect(args):
+    """Collect data using deb build from current tree.
+
+    @param args: cmdline args
+    @return_value: fail count
+    """
+    failed = 0
+    tmpdir = util.TempDir(tmpdir=args.data_dir, preserve=args.preserve_data)
+
+    with tmpdir as data_dir:
+        args.data_dir = data_dir
+        args.deb = os.path.join(tmpdir.tmpdir, 'cloud-init_all.deb')
+        try:
+            failed += bddeb.bddeb(args)
+            failed += collect.collect(args)
+        except Exception:
+            failed += 1
+            raise
+
+    return failed
+
+
+def tree_run(args):
+    """Run test suite using deb build from current tree.
+
+    @param args: cmdline args
+    @return_value: fail count
+    """
+    failed = 0
+    tmpdir = util.TempDir(tmpdir=args.data_dir, preserve=args.preserve_data)
+
+    with tmpdir as data_dir:
+        args.data_dir = data_dir
+        args.deb = os.path.join(tmpdir.tmpdir, 'cloud-init_all.deb')
+        try:
+            failed += bddeb.bddeb(args)
+            failed += collect.collect(args)
+            failed += verify.verify(args)
+        except Exception:
+            failed += 1
+            raise
+
+    return failed
+
+
+def run(args):
+    """Run test suite.
+
+    @param args: cmdline args
+    @return_value: fail count
+    """
+    failed = 0
+    tmpdir = util.TempDir(tmpdir=args.data_dir, preserve=args.preserve_data)
+
+    with tmpdir as data_dir:
+        args.data_dir = data_dir
+        try:
+            failed += collect.collect(args)
+            failed += verify.verify(args)
+        except Exception:
+            failed += 1
+            raise
+
+    return failed
+
+# vi: ts=4 expandtab