summaryrefslogtreecommitdiff
path: root/tests/cloud_tests/verify.py
AgeCommit message (Collapse)Author
2019-02-22tests: integration test failure summary to use traceback if empty errorChad Smith
When integration tests verification fails, the object returned contains has 'error' and 'traceback' keys. Each key can contain empty strings. If the simplified 'error' message is empty, fallback and use the more verbose full 'traceback' text in the failure summary.
2018-10-01tests: allow skipping an entire cloud_test without running.Scott Moser
Individual skipTest or setUp SkipTest will still launch the instance. This allows us to stop the running of the instance so we don't waste cycles or boot systems that are known to fail. Also replace remaining unittest usage in tests/cloud_tests/ with unittest2.
2018-06-12tests: provide human-readable integration test summary when --verboseChad Smith
Integration tests will now provide a brief summary for test failures listed by platform and distribution. The failure summary will only consist of failed test name and assert error message. Drop the verbose dictionary of all integration test output because this content is unreadable given the large number of integration test results listed within this dictionary.
2018-03-16cc_snap: Add new module to install and configure snapd and snap packages.Chad Smith
Support installing and configuring snaps on ubuntu systems. Now, cloud-config files can provide a list or dictionary of snap:assertions which will be allow configuration of snapd on a system via 'snap ack' calls. The snap:commands configuration option supports arbitrary system commands intended to interact with snappy's cli. This allows users to run arbitrary snappy commands to create users, download, install and configure snap packages and snapd. This branch also deprecates old snappy and snap_config modules leaving warnings in documentation and runtime for consumers of these modules. Deprecated snap* modules will be dropped in cloud-init v.18.2 release.
2018-01-25tests: Collect script output as binary, collect systemd journal, fix lxd.Scott Moser
This adds collection a gzip compressed systemd journal on systemd systems. The file can later be reviewed with:   zcat system.journal.gz > system.journal   journalctl --file=system.journal [-o short-monotonic ..] To support this:   * modify test harness infrastructure to not assume content is utf-8.   * fix lxd platform to support make '_execute' return bytes rather     than a string. https://github.com/lxc/pylxd/issues/268 Also switched the base collectors to use /bin/sh as others already did.
2017-06-08Integration Testing: tox env, pyxld 2.2.3, and revamp frameworkWesley Wiedenmeier
Massive update to clean up and greatly enhance the integration testing framework developed by Wesley Wiedenmeier. - Updated tox environment to run integration test 'citest' to utilize pylxd 2.2.3 - Add support for distro feature flags - add framework for feature flags to release config with feature groups and overrides allowed in any release conf override level - add support for feature flags in platform and config handling - during collect, skip testcases that require features not supported by the image with a warning message - Enable additional distros (i.e. centos, debian) - Add 'bddeb' command to build a deb from the current working tree cleanly in a container, so deps do not have to be installed on host - Adds a command line option '--preserve-data' that ensures that collected data will be left after tests run. This also allows the directory to store collected data in during the run command to be specified using '--data-dir'. - Updated Read the Docs testing page and doc strings for pep 257 compliance
2017-04-21pylint: fix all logging warningsJoshua Powers
This will change all instances of LOG.warn to LOG.warning as warn is now a deprecated method. It will also make sure any logging uses lazy logging by passing string format arguments as function parameters.
2016-12-22integration test: initial commit of integration test frameworkWesley Wiedenmeier
The adds in end-to-end testing of cloud-init. The framework utilizes LXD and cloud images as a backend to test user-data passed in. Arbitrary data is then captured from predefined commands specified by the user. After collection, data verification is completed by running a series of Python unit tests against the collected data. Currently only the Ubuntu Trusty, Xenial, Yakkety, and Zesty releases are supported. Test cases for 50% of the modules is complete and available. Additionally a Read the Docs file was created to guide test writing and execution.