summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
Diffstat
-rw-r--r--docs/_include/common-references.txt4
-rw-r--r--docs/contributing/development.rst10
-rw-r--r--docs/contributing/documentation.rst50
-rw-r--r--docs/contributing/index.rst1
-rw-r--r--docs/documentation.rst43
-rw-r--r--docs/index.rst3
-rw-r--r--docs/testing.rst207
7 files changed, 258 insertions, 60 deletions
diff --git a/docs/_include/common-references.txt b/docs/_include/common-references.txt
index de4f76e7..a921ec67 100644
--- a/docs/_include/common-references.txt
+++ b/docs/_include/common-references.txt
@@ -5,5 +5,7 @@
.. _Phabricator: https://phabricator.vyos.net/
.. _802.1ad: https://en.wikipedia.org/wiki/IEEE_802.1ad
.. _802.1q: https://en.wikipedia.org/wiki/IEEE_802.1Q
+.. _`VyOS CI`: https://ci.vyos.net
+.. _vyos-build: https://github.com/vyos/vyos-build
-.. start_vyoslinter \ No newline at end of file
+.. start_vyoslinter
diff --git a/docs/contributing/development.rst b/docs/contributing/development.rst
index cf274a33..591deedf 100644
--- a/docs/contributing/development.rst
+++ b/docs/contributing/development.rst
@@ -295,7 +295,7 @@ device if you happen to be a crazy scientist.
conf = config
else:
conf = Config()
-
+
# Base path to CLI nodes
base = ['...', '...']
# Convert the VyOS config to an abstract internal representation
@@ -320,7 +320,7 @@ device if you happen to be a crazy scientist.
c = get_config()
verify(c)
generate(c)
- apply(c)
+ apply(c)
except ConfigError as e:
print(e)
sys.exit(1)
@@ -685,9 +685,9 @@ Migrating old CLI
Continuous Integration
======================
-VyOS makes use of Jenkins_ as our Continuous Integration (CI) service. Our CI
-server is publicly accessible here: https://ci.vyos.net. You can get a brief
-overview of all required components shipped in a VyOS ISO.
+VyOS makes use of Jenkins_ as our Continuous Integration (CI) service. Our
+`VyOS CI`_ server is publicly accessible here: https://ci.vyos.net. You can get
+a brief overview of all required components shipped in a VyOS ISO.
To build our modules we utilize a CI/CD Pipeline script. Each and every VyOS
component comes with it's own ``Jenkinsfile`` which is (more or less) a copy.
diff --git a/docs/contributing/documentation.rst b/docs/contributing/documentation.rst
deleted file mode 100644
index eb8db3e1..00000000
--- a/docs/contributing/documentation.rst
+++ /dev/null
@@ -1,50 +0,0 @@
-.. _contrib-documentation:
-
-Documentation
--------------
-
-VyOS documentation is written in reStructuredText and generated to Read the Docs
-pages with Sphinx, as per the Python tradition, as well as PDF files for offline
-use through LaTeX.
-
-We welcome all sorts of contributions to the documentation. Not just
-new additions but also corrections to existing documentation.
-
-Guidelines
-^^^^^^^^^^
-
-There are a few things to keep in mind when contributing to the
-documentation, for the sake of consistency and readability.
-
-Take a look at the :doc:`/documentation` page for an intricate explanation
-of the documentation process.
-
-The following is a quick summary of the rules:
-
-- Use American English at all times. It's always a good idea to run
- your text through a grammar and spell checker, such as `Grammarly`_.
-- Don't forget to update ``index.rst`` when adding a new node.
-- Try not to exceed 80 characters per line, but don't break URLs over this.
-- Properly quote commands, filenames and brief code snippets with double backticks.
-- Use literal blocks for longer snippets.
-- Leave a newline before and after a header.
-- Indent with two spaces.
-- When in doubt, follow the style of existing documentation.
-
-And finally, remember that the reStructuredText files aren't
-exclusively for generating HTML and PDF. They should be human-readable
-and easily perused from a console.
-
-Building
-^^^^^^^^
-
-The source is kept in the Git repository
-https://github.com/vyos/vyos-documentation
-
-You can follow the instructions in the README to build and test your changes.
-
-You can either install Sphinx (and TeX Live for PDF output) and build the
-documentation locally, or use the `Dockerfile`_ to build it in a container.
-
-.. _Dockerfile: https://github.com/vyos/vyos-documentation/blob/master/docker/Dockerfile
-.. _Grammarly: https://www.grammarly.com/
diff --git a/docs/contributing/index.rst b/docs/contributing/index.rst
index 481c6da8..a5942f96 100644
--- a/docs/contributing/index.rst
+++ b/docs/contributing/index.rst
@@ -7,6 +7,5 @@ Contributing
build-vyos
development
- documentation
issues-features
upstream-packages
diff --git a/docs/documentation.rst b/docs/documentation.rst
index 849008a8..5161f773 100644
--- a/docs/documentation.rst
+++ b/docs/documentation.rst
@@ -15,6 +15,43 @@ guide how to do so.
you open up a Phabricator_ task prior to submitting a Pull-Request to the
documentation.
+VyOS documentation is written in reStructuredText and generated to Read the Docs
+pages with Sphinx, as per the Python tradition, as well as PDF files for offline
+use through LaTeX. We welcome all sorts of contributions to the documentation.
+Not just new additions but also corrections to existing documentation.
+
+The documentation source is kept in the Git repository at
+https://github.com/vyos/vyos-documentation and you can follow the instructions
+in the README.md_ to build and test your changes.
+
+You can either install Sphinx (and TeX Live for PDF output) and build the
+documentation locally, or use the Dockerfile_ to build it in a container.
+
+Guidelines
+==========
+
+There are a few things to keep in mind when contributing to the
+documentation, for the sake of consistency and readability.
+
+Take a look at the :doc:`/documentation` page for an intricate explanation
+of the documentation process.
+
+The following is a quick summary of the rules:
+
+- Use American English at all times. It's always a good idea to run
+ your text through a grammar and spell checker, such as `Grammarly`_.
+- Don't forget to update ``index.rst`` when adding a new node.
+- Try not to exceed 80 characters per line, but don't break URLs over this.
+- Properly quote commands, filenames and brief code snippets with double backticks.
+- Use literal blocks for longer snippets.
+- Leave a newline before and after a header.
+- Indent with two spaces.
+- When in doubt, follow the style of existing documentation.
+
+And finally, remember that the reStructuredText files aren't
+exclusively for generating HTML and PDF. They should be human-readable
+and easily perused from a console.
+
Forking Workflow
================
@@ -36,7 +73,7 @@ access to the official codebase.
* Fork this project on GitHub https://github.com/vyos/vyos-documentation/fork
-* Clone fork to local machine, then change to that directory
+* Clone fork to local machine, then change to that directory
``$ cd vyos-documentation``
* Install the requirements ``$ pip install -r requirements.txt``
@@ -281,7 +318,7 @@ URL. This is heavily used in the :ref:`release-notes` section.
Page content
------------
-The documentation have 3 different types of pages, the same kind of pages must
+The documentation have 3 different types of pages, the same kind of pages must
have the same structure to achieve a recognition factor.
All RST files must follow the same TOC Level syntax and have to start with
@@ -342,6 +379,8 @@ predefined structure.
.. _reStructuredText: http://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html
.. _reStructuredTextDirectives: https://docutils.sourceforge.io/docs/ref/rst/directives.html
.. _README.md: https://github.com/vyos/vyos-documentation/blob/master/README.md
+.. _Dockerfile: https://github.com/vyos/vyos-documentation/blob/master/docker/Dockerfile
+.. _Grammarly: https://www.grammarly.com/
.. include:: /_include/common-references.txt
.. start_vyoslinter
diff --git a/docs/index.rst b/docs/index.rst
index 574a2c49..f81adfc8 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -20,7 +20,7 @@ VyOS User Guide
installation/index
quick-start
cli
-
+
.. toctree::
:maxdepth: 2
:includehidden:
@@ -41,6 +41,7 @@ VyOS User Guide
contributing/index
debugging
+ testing
documentation
coverage
copyright
diff --git a/docs/testing.rst b/docs/testing.rst
new file mode 100644
index 00000000..b054a646
--- /dev/null
+++ b/docs/testing.rst
@@ -0,0 +1,207 @@
+.. _testing:
+
+#######
+Testing
+#######
+
+One of the major advantages introduced in VyOS 1.3 is an autmated test framework.
+When assembling an ISO image multiple things can go wrong badly and publishing
+a faulty ISO makes no sense. The user is disappointed by the quality of the image
+and the developers get flodded with bug reports over and over again.
+
+As the VyOS documentation is not only for users but also for the developers -
+and we keep no secret documentation - this section describes how the automated
+testing works.
+
+Jenkins CI
+==========
+
+Our `VyOS CI`_ system is based on Jenkins and builds all our required packages
+for VyOS 1.2 to 1.4. In addition to the package build, there is the vyos-build
+Job which builds and tests the VyOS ISO image which is published after a
+successfull test drive.
+
+We differentiate in two independent tests, which are both run in parallel by
+two separate QEmu instances which are launched via ``make test`` and ``make
+testc`` from within the vyos-build_ repository.
+
+Smoketests
+==========
+
+Smoketests executes predefined VyOS CLI commands and checks if the desired
+daemon/service configuration is rendert - that is how to put it "short".
+
+When and ISO image is assembled by the `VyOS CI`_, the ``BUILD_SMOKETEST``
+parameter is enabled by default, which will extend the ISO configuration line
+with the following packages:
+
+.. code-block:: python
+
+ def CUSTOM_PACKAGES = ''
+ if (params.BUILD_SMOKETESTS)
+ CUSTOM_PACKAGES = '--custom-package vyos-1x-smoketest'
+
+So if you plan to build your own custom ISO image and wan't to make use of our
+smoketests, ensure that you have the `vyos-1x-smoketest` package installed.
+
+The ``make test`` command from the vyos-build_ repository will launch a new
+QEmu instance and the ISO image is first installed to the virtual harddisk.
+
+After its first boot into the newly installed system the main Smoketest script
+is executed, it can be found here: `/usr/bin/vyos-smoketest`
+
+The script only searches for executable "test-cases" under
+``/usr/libexec/vyos/tests/smoke/cli/`` and executes them one by one.
+
+.. note:: As Smoketests will alter the system configuration and you are logged
+ in remote you may loose your connection to the system.
+
+Manual Smoketest Run
+--------------------
+
+On the other hand - as each test is contain in its own file - one can always
+execute a single Smoketest by hand by simply running the Python test scripts.
+
+Example:
+
+.. code-block:: none
+
+ vyos@vyos:~$ /usr/libexec/vyos/tests/smoke/cli/test_protocols_bgp.py
+ test_bgp_01_simple (__main__.TestProtocolsBGP) ... ok
+ test_bgp_02_neighbors (__main__.TestProtocolsBGP) ... ok
+ test_bgp_03_peer_groups (__main__.TestProtocolsBGP) ... ok
+ test_bgp_04_afi_ipv4 (__main__.TestProtocolsBGP) ... ok
+ test_bgp_05_afi_ipv6 (__main__.TestProtocolsBGP) ... ok
+ test_bgp_06_listen_range (__main__.TestProtocolsBGP) ... ok
+ test_bgp_07_l2vpn_evpn (__main__.TestProtocolsBGP) ... ok
+ test_bgp_08_zebra_route_map (__main__.TestProtocolsBGP) ... ok
+ test_bgp_09_distance_and_flowspec (__main__.TestProtocolsBGP) ... ok
+ test_bgp_10_vrf_simple (__main__.TestProtocolsBGP) ... ok
+ test_bgp_11_confederation (__main__.TestProtocolsBGP) ... ok
+ test_bgp_12_v6_link_local (__main__.TestProtocolsBGP) ... ok
+ test_bgp_13_solo (__main__.TestProtocolsBGP) ... ok
+
+ ----------------------------------------------------------------------
+ Ran 13 tests in 348.191s
+
+ OK
+
+Interface based tests
+---------------------
+
+Our smoketests not only test daemons and serives, but also check if what we
+configure for an interface works. Thus there is a common base classed named:
+``base_interfaces_test.py`` which holds all the common code that an interface
+supports and is tested.
+
+Those common tests consists out of:
+
+* Add one or more IP addresses
+* DHCP client and DHCPv6 prefix delegation
+* MTU size
+* IP and IPv6 options
+* Port description
+* Port disable
+* VLANs (QinQ and regular 802.1q)
+* ...
+
+.. note:: When you are working on interface configuration and you also wan't to
+ test if the Smoketests pass you would normally loose the remote SSH connection
+ to your :abbr:`DUT (Device Under Test)`. To handle this issue, some of the
+ interface based tests can be called with an environment variable beforehand
+ to limit the number of interfaces used in the test. By default all interface
+ e.g. all Ethernet interfaces are used.
+
+.. code-block:: none
+
+ vyos@vyos:~$ TEST_ETH="eth1 eth2" /usr/libexec/vyos/tests/smoke/cli/test_interfaces_bonding.py
+ test_add_multiple_ip_addresses (__main__.BondingInterfaceTest) ... ok
+ test_add_single_ip_address (__main__.BondingInterfaceTest) ... ok
+ test_bonding_hash_policy (__main__.BondingInterfaceTest) ... ok
+ test_bonding_lacp_rate (__main__.BondingInterfaceTest) ... ok
+ test_bonding_min_links (__main__.BondingInterfaceTest) ... ok
+ test_bonding_remove_member (__main__.BondingInterfaceTest) ... ok
+ test_dhcpv6_client_options (__main__.BondingInterfaceTest) ... ok
+ test_dhcpv6pd_auto_sla_id (__main__.BondingInterfaceTest) ... ok
+ test_dhcpv6pd_manual_sla_id (__main__.BondingInterfaceTest) ... ok
+ test_interface_description (__main__.BondingInterfaceTest) ... ok
+ test_interface_disable (__main__.BondingInterfaceTest) ... ok
+ test_interface_ip_options (__main__.BondingInterfaceTest) ... ok
+ test_interface_ipv6_options (__main__.BondingInterfaceTest) ... ok
+ test_interface_mtu (__main__.BondingInterfaceTest) ... ok
+ test_ipv6_link_local_address (__main__.BondingInterfaceTest) ... ok
+ test_mtu_1200_no_ipv6_interface (__main__.BondingInterfaceTest) ... ok
+ test_span_mirror (__main__.BondingInterfaceTest) ... ok
+ test_vif_8021q_interfaces (__main__.BondingInterfaceTest) ... ok
+ test_vif_8021q_lower_up_down (__main__.BondingInterfaceTest) ... ok
+ test_vif_8021q_mtu_limits (__main__.BondingInterfaceTest) ... ok
+ test_vif_8021q_qos_change (__main__.BondingInterfaceTest) ... ok
+ test_vif_s_8021ad_vlan_interfaces (__main__.BondingInterfaceTest) ... ok
+ test_vif_s_protocol_change (__main__.BondingInterfaceTest) ... ok
+
+ ----------------------------------------------------------------------
+ Ran 23 tests in 244.694s
+
+ OK
+
+This will limit the `bond` interface test to only make use of `eth1` and `eth2`
+as member ports.
+
+Config Load Tests
+=================
+
+The other part of our tests are called "config load tests". The config load tests
+will load - one after another - arbitrary configuration files to test if the
+configuration migration scripts work as designed and that a given set of
+functionality still can be loaded with a fresh VyOS ISO image.
+
+The configurations are all derived from production systems and can not only act
+as a testcase but also as reference if one wants to enable a certain feature.
+The configurations can be found here:
+https://github.com/vyos/vyos-1x/tree/current/smoketest/configs
+
+The entire test is controlled by the main wrapper script ``/usr/bin/vyos-configtest``
+which behaves in the same way as the main smoketest script. It scans the folder
+for potential configuration files and issues a ``load`` command one after another.
+
+Manual config load test
+-----------------------
+
+One is not bound to load all configurations one after another but can also load
+individual test configurations on his own.
+
+.. code-block:: none
+
+ vyos@vyos:~$ configure
+ load[edit]
+
+ vyos@vyos# load /usr/libexec/vyos/tests/config/ospf-small
+ Loading configuration from '/usr/libexec/vyos/tests/config/ospf-small'
+ Load complete. Use 'commit' to make changes effective.
+ [edit]
+ cpo@LR1.wue3# compare
+ [edit interfaces ethernet eth0]
+ -hw-id 00:50:56:bf:c5:6d
+ [edit interfaces ethernet eth1]
+ +duplex auto
+ -hw-id 00:50:56:b3:38:c5
+ +speed auto
+ [edit interfaces]
+ -ethernet eth2 {
+ - hw-id 00:50:56:b3:9c:1d
+ -}
+ -vti vti1 {
+ - address 192.0.2.1/30
+ -}
+ ...
+
+ vyos@vyos# commit
+ vyos@vyos#
+
+.. note:: Some of the configurations have preconditions which need to be met.
+ Those most likely include generation of crypographic keys before the config
+ can be applied - you will get a commit error otherwise. If you are interested
+ how those preconditions are fulfilled check the vyos-build_ repository and
+ the ``scripts/check-qemu-install`` file.
+
+.. include:: /_include/common-references.txt
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-30 09:17:31 +0000