From aa0f5b38aec14428b4b80e06f90ff781f8bca5f1 Mon Sep 17 00:00:00 2001 From: Rene Mayrhofer Date: Mon, 22 May 2006 05:12:18 +0000 Subject: Import initial strongswan 2.7.0 version into SVN. --- doc/src/makecheck.html | 684 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 684 insertions(+) create mode 100644 doc/src/makecheck.html (limited to 'doc/src/makecheck.html') diff --git a/doc/src/makecheck.html b/doc/src/makecheck.html new file mode 100644 index 000000000..7fa3a3bcb --- /dev/null +++ b/doc/src/makecheck.html @@ -0,0 +1,684 @@ + + +FreeS/WAN "make check" guide + + + + + + + + +

How to configure to use "make check"

+ +

What is "make check"

+"make check" is a target in the top level makefile. It takes care of +running a number of unit and system tests to confirm that FreeSWAN has +been compiled correctly, and that no new bugs have been introduced. +

+As FreeSWAN contains both kernel and userspace components, doing testing +of FreeSWAN requires that the kernel be simulated. This is typically difficult +to do as a kernel requires that it be run on bare hardware. A technology +has emerged that makes this simpler. This is +User Mode Linux. +

+ +

+User-Mode Linux is a way to build a Linux kernel such that it can run as a process +under another Linux (or in the future other) kernel. Presently, this can only be +done for 2.4 guest kernels. The host kernel can be 2.2 or 2.4. +

+ +

+"make check" expects to be able to build User-Mode Linux kernels with FreeSWAN included. +To do this it needs to have some files downloaded and extracted prior +to running "make check". This is described in the +UML testing document. +

+ +

+After having run the example in the UML testing document and +successfully brought up the four machine combination, you are ready to +use "make check" +

+ +

Running "make check"

+"make check" works by walking the FreeSWAN source tree invoking the +"check" target at each node. At present there are tests defined only +for the klips directory. These tests will use the UML +infrastructure to test out pieces of the klips code. +

+The results of the tests can be recorded. If the environment variable +$REGRESSRESULTS is non-null, then the results of each +test will be recorded. This can be used as part of a nightly +regression testing system, see +Nightly testing for more details. +

+"make check" otherwise prints a minimal amount of output for each +test, and indicates pass/fail status of each test as they are run. +Failed tests do not cause failure of the target in the form of exit +codes. +

+ +

How to write a "make check" test

+ +

Structure of a test

+ +

+Each test consists of a set of directories under testing/. +There are directories for klips, pluto, packaging +and libraries. +Each directory has a list of tests to run is stored in a file called TESTLIST in that directory. e.g. testing/klips/TESTLIST. +

+ +

The TESTLIST

+This isn't actually a shell script. It just looks like one. Some tools other than +/bin/sh process it. Lines that start with # are comments.

+ +

+# test-kind     directory-containing-test       expectation     [PR#]
+

+ +

The first word provides the test type, detailed below.

The second word is the name of the test to run. This the directory +in which the test case is to be found..

The third word may be one of: +

blank/good: the test is believed to function, report failure
bad: the test is known to fail, report unexpected success
suspended: the test should not be run

+ +

+The fourth word may be a number, which is a PR# if the test is +failing. +

+ +

Test kinds

+The test types are: + +

skiptest: means run no test.
ctltest: means run a single system without input/output.
klipstest: means run a single system with input/output networks
umlplutotest: means run a pair of systems
umlXhost: run an arbitrary number of systems +
suntest (TBD): means run a quad of east/west/sunrise/sunset
roadtest (TBD): means run a trio of east-sunrise + warrior
extrudedtest (TBD): means run a quad of east-sunrise + warriorsouth + park
mkinsttest +: a test of the "make install" machinery.
kernel_test_patch +: a test of the "make kernelpatch" machinery.

+ +Tests marked (TBD) have yet to be fully defined. +

+ +

+Each test directory has a file in it called testparams.sh. +This file sets a number of environment variables to define the +parameters of the test. +

+ +

Common parameters

TESTNAME

the name of the test (repeated for checking purposes)

TEST_TYPE

the type of the test (repeat of type type above)

TESTHOST

the name of the UML machine to run for the test, typically "east" + or "west"

TEST_PURPOSE

The purpose of the test is one of: + +

goal: The goal purpose is where a test is defined for code that is not yet +finished. The test indicates when the goals have in fact been reached.
regress: This is a test to determine that a previously existing bug has been repaired. This +test will initially be created to reproduce the bug in isolation, and then the bug will +be fixed.
exploit: This is a set of packets/programs that causes a vulnerability to be +exposed. It is a specific variation of the regress option.

TEST_GOAL_ITEM

+

in the case of a goal test, this is a reference to the requirements document

TEST_PROB_REPORT

in the case of regression test, this the problem report number from GNATS

TEST_EXPLOIT_URL

in the case of an exploit, this is a URL referencing the paper explaining +the origin of the test and the origin of exploit software

REF_CONSOLE_OUTPUT

a file in the test directory that contains the sanitized console + output against which to compare the output of the actual test.

REF_CONSOLE_FIXUPS

a list of scripts (found in klips/test/fixups) to + apply to sanitize the console output of the machine under test. + These are typically perl, awk or sed scripts that remove things in + the kernel output that change each time the test is run and/or + compiled. +

INIT_SCRIPT

a file of commands that is fed into the virtual machine's console + in single user mode prior to starting the tests. This file will + usually set up any eroute's and SADB entries that are required for + the test.

Lines beginning with # are skipped. Blank lines are + skipped. Otherwise, a shell prompted is waited for each time + (consisting of \n#) and then the command is sent. + Note that the prompt is waited for before the command and not after, + so completion of the last command in the script is not + required. This is often used to invoke a program to monitor the + system, e.g. ipsec pf_key. +

RUN_SCRIPT

a file of commands that is fed into the virtual machine's console + in single user mode, before the packets are sent. On single machine + tests, this script doesn't provide any more power than INIT_SCRIPT, + but is implemented for consistency's sake.

FINAL_SCRIPT

a file of commands that is fed into the virtual machine's console + in single user mode after the final packet is sent. Similar to INIT_SCRIPT, + above. If not specified, then the single command "halt" is sent. + If specified, then the script should end with a halt command to + nicely shutdown the UML. +

CONSOLEDIFFDEBUG

If set to "true" then the series of console fixups (see REF_CONSOLE_FIXUPS) will be output after it is constructed. (It should be set to "false", or unset otherwise)

NETJIGDEBUG

If set to "true" then the series of console fixups (see REF_CONSOLE_FIXUPS) will be output after it is constructed. (It should be set to "false", or unset otherwise)

NETJIGTESTDEBUG

If set to "netjig", then the results of talking to the uml_netjig +will be printed to stderr during the test. In addition, the jig will +be invoked with --debug, which causes it to log its process ID, and +wait 60 seconds before continuing. This can be used if you are trying +to debug the uml_netjig program itself. +

HOSTTESTDEBUG

If set to "hosttest", then the results of taling to the consoles of the UMLs will +be printed to stderr during the test. +

NETJIGWAITUSER

If set to "waituser", then the scripts will wait forever for user + input before they shut the tests down. Use this is if you are + debugging through the kernel.

PACKETRATE

A number, in miliseconds (default is 500ms) at which packets will + be replayed by the netjig.

+ + +

KLIPStest paramaters

+The klipstest function starts a program +(testing/utils/uml_netjig/uml_netjig) to +setup a bunch of I/O sockets (that simulate network interfaces). It +then exports the references to these sockets to the environment and +invokes (using system()) a given script. It waits for the script to +finish. +

+ + + +

+The script invoked (testing/utils/host-test.tcl) is a TCL +expect script that arranges to start the UML +and configure it appropriately for the test. The configuration is done +with the script given above for INIT_SCRIPT. The TCL script then forks, +leaves the UML in the background and exits. uml_netjig continues. It then +starts listening to the simulated network answering ARPs and inserting +packets as appropriate. +

+ +

+The klipstest function invokes uml_netjig with arguments +to capture output from network interface(s) and insert packets as +appropriate: +

PUB_INPUT: a pcap file to feed in on + the public (encrypted) interface. (typically, eth1)
PRIV_INPUT: a pcap file to feed in on the private (plain-text) interface + (typically, eth0).
REF_PUB_OUTPUT: a text file containing tcpdump output. Packets on the public + (eth1) interface are captured to a + pcap file by + uml_netjig. The klipstest function then uses tcpdump on + the file to produce text output, which is compared to the file given.
REF_PUB_FILTER: a program that will filter the TCPDUMP output to do further processing. Defaults to "cat".
REF_PRIV_OUTPUT: a text file containing tcpdump output. Packets on the private + (eth0) interface are captured and compared after conversion by + tcpdump, as with REFPUBOUTPUT.
REF_PRIV_FILTER: a program that will filter the TCPDUMP output to do further processing. Defaults to "cat".
EXITONEMPTY: a flag for uml_netjig. It should contain + "--exitonempty" of uml_netjig should exit when all of the input + (PUBINPUT,PRIVINPUT) packets have been injected.
ARPREPLY: a flag for uml_netjig. It should contain "--arpreply" + if uml_netjig should reply to ARP requests. One will + typically set this to avoid having to fudge the ARP cache manually.
TCPDUMPFLAGS: a set of flags for the tcpdump used when converting captured + output. Typical values will include "-n" to turn off DNS, and often + "-E" to set the decryption key (tcpdump 3.7.1 and higher only) for + ESP packets. The "-t" flag (turn off timestamps) is provided automatically
NETJIG_EXTRA: additional comments to be sent to the netjig. This may arrange to + record or create additional networks, or may toggle options. +

+ +

mkinsttest paramaters

+The basic concept of the mkinsttest test type is that it +performs a "make install" to a temporary $DESTDIR. The resulting tree can then +be examined to determine if it was done properly. The files can be uninstalled +to determine if the file list was correct, or the contents of files can be +examined more precisely. +

+ +

INSTALL_FLAGS

If set, then an install will be done. This provides the set of flags to +provide for the install. The target to be used (usually "install") must be +among the flags.

POSTINSTALL_SCRIPT

If set, a script to run after initial "make install". Two arguments are provided: an absolute path to the root of the FreeSWAN src tree, and an absolute path to the temporary installation area.

INSTALL2_FLAGS

If set, a second install will be done using these flags. Similarly to +INSTALL_FLAGS, the target must be among the flags.

UNINSTALL_FLAGS

If set, an uninstall will be done using these flags. Similarly to +INSTALL_FLAGS, the target (usually "uninstall") must be among the flags.

REF_FIND_f_l_OUTPUT

If set, a find $ROOT ( -type f -or -type -l ) will be done to get a list of a real files and symlinks. The resulting file will be compared +to the file listed by this option.

REF_FILE_CONTENTS

If set, it should point to a file containing records for the form: +

+  reffile   samplefile
+

+one record per line. A diff between the provided reference file, and the +sample file (located in the temporary installation root) will be done for +each record. +

+ +

rpm_build_install_test paramaters

+The rpm_build_install_test type is to verify that the proper +packing list is produced by "make rpm", and that the mechanisms for +building the kernel modules produce consistent results. +

+ +

RPM_KERNEL_SOURCE: Point to an extracted copy of the RedHat kernel source code. Variables +from the environment may be used.
REF_RPM_CONTENTS: This is a file containing one record per line. Each record consists +of a RPM name (may contain wildcards) and a filename to compare the +contents to. The RPM will be located and a file list will be produced with +rpm2cpio.

+ +

libtest paramaters

+The libtest test is for testing library routines. The library file is +expected to provided an #ifdef by the name of +library. +The libtest type invokes the C compiler to compile this file, links it against +libfreeswan.a (to resolve any other dependancies) and runs the +test with the -r argument to invoke a regression test.

The library test case is expected to do a self-test, exiting with status code 0 if everything is okay, and with non-zero otherwise. A core dump (exit code greater than 128) is noted specifically. +

+Unlike other tests, there are no subdirectories required, or other +parameters to set. +

+ +

umlplutotest paramaters

+The umlplutotest function starts a pair of user mode line processes. +This is a 2-host version of umlXhost. The "EAST" and "WEST" slots are defined. +

+ +

umlXhost parameters

+The umlXtest function starts an arbitrary number of user mode line processes. +

+ + + +

+The script invoked (testing/utils/Xhost-test.tcl) is a TCL +expect script that arranges to start each +UML +and configure it appropriately for the test. It then starts listening +(using uml_netjig) to the simulated network answering ARPs and +inserting packets as appropriate. +

+ +

+umlXtest has a series of slots, each of which should be filled by a host. +The list of slots is controlled by the variable, XHOST_LIST. This variable +should be set to a space seperated list of slots. The former umlplutotest +is now implemented as a variation of the umlXhost test, +with XHOST_LIST="EAST WEST". +

+ +

+For each host slot that is defined, a series of variables should be +filled in, defining what configuration scripts to use for that host. +

+ +

+The following are used to control the console input and output to the system. +Where the string ${host} is present, the host slot should be filled in. +I.e. for the two host system with XHOST_LIST="EAST WEST", then the +variables: EAST_INIT_SCRIPT and WEST_INIT_SCRIPT will exist. +

${host}HOST: The name of the UML host which will fill this slot
${host}_INIT_SCRIPT: a file of commands that is fed into the virtual machine's console + in single user mode prior to starting the tests. This file will + usually set up any eroute's and SADB entries that are required for + the test. Similar to INIT_SCRIPT, above.
+
${host}_RUN_SCRIPT: a file of commands that is fed into the virtual machine's console + in single user mode, before the packets are sent. This set of + commands is run after all of the virtual machines are initialized. + I.e. after EAST_INIT_SCRIPT AND WEST_INIT_SCRIPT. This script + can therefore do things that require that all machines are properly + configured.
+
${host}_RUN2_SCRIPT: a file of commands that is fed into the virtual machine's console + in single user mode, after the packets are sent. This set of + commands is run before any of the virtual machines have been shut + down. (I.e. before EAST_FINAL_SCRIPT AND WEST_FINAL_SCRIPT.) + This script can therefore catch post-activity status reports.
+
${host}_FINAL_SCRIPT: a file of commands that is fed into the virtual machine's console + in single user mode after the final packet is sent. Similar to INIT_SCRIPT, + above. If not specified, then the single command "halt" is sent. Note that + when this script is run, the other virtual machines may already have been killed. + If specified, then the script should end with a halt command to nicely + shutdown the UML. +
+
REF_${host}_CONSOLE_OUTPUT: Similar to REF_CONSOLE_OUTPUT, above. +

+ +

Some additional flags apply to all hosts: +

REF_CONSOLE_FIXUPS: a list of scripts (found in klips/test/fixups) to + apply to sanitize the console output of the machine under test. + These are typically perl, awk or sed scripts that remove things in + the kernel output that change each time the test is run and/or + compiled. +

+ +

In addition to input to the console, the networks may have input +fed to them: +

EAST_INPUT/WEST_INPUT: a pcap file to feed in on + the private network side of each network. The "EAST" and "WEST" here +refer to the networks, not the hosts.
REF_PUB_FILTER: a program that will filter the TCPDUMP output to do further processing. Defaults to "cat".
REF_EAST_FILTER/REF_WEST_FILTER: a program that will filter the TCPDUMP output to do further processing. Defaults to "cat".
TCPDUMPFLAGS: a set of flags for the tcpdump used when converting captured + output. Typical values will include "-n" to turn off DNS, and often + "-E" to set the decryption key (tcpdump 3.7.1 and higher only) for + ESP packets. The "-t" flag (turn off timestamps) is provided automatically
REF_EAST_OUTPUT/REF_WEST_OUTPUT: a text file containing tcpdump output. Packets on the private + (eth0) interface are captured and compared after conversion by + tcpdump, as with REF_PUB_OUTPUT.
HOSTTESTDEBUG=hosttest export HOSTTESTDEBUG: This will show all interactions with the user-mode-linux + consoles

+ +

kernel_patch_test paramaters

+The kernel_patch_test function takes some kernel source, copies it with +lndir, and then applies the patch as produced by "make kernelpatch". +

+The following are used to control the input and output to the system: +

KERNEL_NAME: the kernel name, typically something like "linus" or "rh"
KERNEL_VERSION: the kernel version number, as in "2.2" or "2.4".
KERNEL_${KERNEL_NAME}${KERNEL_VERSION}_SRC: This variable should set in the environment, probably in + ~/freeswan-regress-env.sh. Examples of this variables would be + KERNEL_LINUS2_0_SRC or KERNEL_RH7_3_SRC. This variable should point + to an extracted copy of the kernel source in question.
REF_PATCH_OUTPUT: a copy of the patch output to compare against
KERNEL_PATCH_LEAVE_SOURCE: If set to a non-empty string, then the patched kernel source is not + removed at the end of the test. This will typically be set in the + environment while debugging.

+ +

module_compile paramaters

+The module_compile test attempts to build the KLIPS module against a +given set of kernel source. This is also done by the RPM tests, but +in a very specific manner. +

+There are two variations of this test - one where the kernel either +doesn't need to be configured, or is already done, and tests were there +is a local configuration file. +

+Where the kernel doesn't need to be configured, the kernel source that +is found is simply used. It may be a RedHat-style kernel, where one +can cause it to configure itself via rhconfig.h-style definitions. Or, +it may just be a kernel tree that has been configured. +

+If the variable KERNEL_CONFIG_FILE is set, then a new directory is +created for the kernel source. It is populated with lndir(1). The referenced +file is then copied in as .config, and "make oldconfig" is used to configure +the kernel. This resulting kernel is then used as the reference source. +

+In all cases, the kernel source is found the same was for the kernelpatch +test, i.e. via KERNEL_VERSION/KERNEL_NAME and KERNEL_${KERNEL_NAME}${KERNEL_VERSION}_SRC.

+Once there is kernel source, the module is compiled using the top-level +"make module" target. +

+The test is considered successful if an executable is found in OUTPUT/module/ipsec.o at the end of the test. +

KERNEL_NAME: the kernel name, typically something like "linus" or "rh"
KERNEL_VERSION: the kernel version number, as in "2.2" or "2.4".
KERNEL_${KERNEL_NAME}${KERNEL_VERSION}_SRC: This variable should set in the environment, probably in + ~/freeswan-regress-env.sh. Examples of this variables would be + KERNEL_LINUS2_0_SRC or KERNEL_RH7_3_SRC. This variable should point + to an extracted copy of the kernel source in question.
KERNEL_CONFIG_FILE: The configuration file for the kernel.
KERNEL_PATCH_LEAVE_SOURCE: If set to a non-empty string, then the configured kernel source is not + removed at the end of the test. This will typically be set in the + environment while debugging.
MODULE_DEF_INCLUDE: The include file that will be used to configure the KLIPS module, and + possibly the kernel source.

+ +

Current pitfalls

+ +

"tcpdump dissector" not available.: This is a non-fatal warning. If uml_netjig is invoked with the -t + option, then it will attempt to use tcpdump's dissector to decode + each packet that it processes. The dissector is presently not + available, so this option it normally turned off at compile time. + The dissector library will be released with tcpdump version + 4.0.

+ + + \ No newline at end of file -- cgit v1.2.3