                        GATED-IDRP AUTOMATED TESTER
                                USER'S GUIDE

Version: 2.41
Last updated: 10/20/97

1.0 Introduction

------------------------------------------------------------------------

1.1 Purpose

This document describes testg, a script written in C-Shell that automates
testing of GateD using the files generated by gen_config.

1.2 Scope

It is assumed that the reader is familiar with gen_config, and general
layout of the testing utilities. The author also assumes that the reader is
familiar with the configuration of GateD, and the basic concepts of
networking.

1.3 Scenario

This program became necessary from the sheer number of different gated.conf
test cases needed for even relatively simple configurations. A basic
four-machine test using 12 slightly different networks and six sub-tests
requires over two hundred different configuration files, and thus as many
separate tests.

Accordingly, this program is designed to minimize the amount of work needed
to test a given physical configuration. It was designed both to be simple
and straightforward to use, and yet flexible enough to use under a wide
variety of tests and conditions.

1.4 Supported Protocols and Software

The current testing suite contains support and tests for BGP, IDRP, RIP, and
OSPF. All versions of GateD are supported at the time of this writing; this
includes gated-pub through 3.6.7, gated-unicast through 4.0.2, and
gated-multicast.

1.5 Definitions

------------------------------------------------------------------------

Configuration file - a file which Gated uses to read in information about a
                     particular machine - its peers, import and export policy,
                     static routes, and operating parameters.  Usually named
                     "gated.conf" or something similar.

Expected results file - a file created by the configuration file generator
                        containing the routes which are expected to be in
                        the routing table following the processing of its
                        related gated.conf file.

Sequence - a particular physical configuration of machines.  A sequence
           defines the machines used, internal and external peers for each
           one, and RDI and RIB information.

Test set - a particular test to apply to each sequence in the test.  Test
           sequences contain import and export information, and which routes
           to include or exclude.

2.0 Using the Program

2.1 Setting up the Program

Testg expects certain environment variables to be set. In order to ease the
setup of the environment a standard configuration is provided by sourceing
"testg_setup.csh".

Testg_setup.csh in turn expects 3 environment variables to be set:

   * GATED_TOOLS_PATH - path to the tools directory this is the parent of
     e.g. scripts which holds testg, gen_config, etc.
   * GATED_TOOLS_HOMEDIR - path to the directory in which test files and
     binaries are to be found.
   * GATED_TOOLS_USE_NAME - if set and not 0 will append the hostname to the
     end of the GATED_TOOLS_HOMEDIR when manipulating working files.

The example tree is as such:

        $GATED_TOOLS_PATH/scripts/{gen_config,route_sets.txt,testg,...}
        $GATED_TOOLS_PATH/templates
        $GATED_TOOLS_PATH/tests

        if GATED_TOOLS_USE_NAME is set and not "0"
                set hostname=`hostname | cut -d . -f 1`
                $GATED_TOOLS_HOMEDIR/$hostname/working/
                $GATED_TOOLS_HOMEDIR/$hostname/log/
        else
                $GATED_TOOLS_HOMEDIR/working/
                $GATED_TOOLS_HOMEDIR/log/

The recommended method is to share (e.g. nfs mount) a single directory
amongst all the test machines. In this directory place the gated build tree
(e.g., in "gated"), the gated tools (e.g., in "gated_tools") and create a
"working" directory. For each architecture to be tested create a symlink to
the obj build dir in the "gated" tree in the "working" tree. Next create a
tree for each machine. Using symlinks make the correct gated binary
accessible from the "working"/$hostname/working directory.

        /net/shared/

        shared/gated/
        shared/gated_tools
        shared/working

        shared/gated/src/obj.BSD_OS-2.0.1-i386
        shared/gated/src/obj.BSD_OS-2.1-i386

        shared/working/obj.BSD_OS-2.0.1-i386 ->
            ../gated/src/obj.BSD_OS-2.0.1-i386
        shared/working/obj.BSD_OS-2.1-i386 ->
            ../gated/src/obj.BSD_OS-2.1-i386

        shared/working/$hostname/working
        shared/working/$hostname/log

        shared/working/<bsd2.1-machine-hostname>/working/gated ->
            ../../obj.BSD_OS-2.1-i386/gated

Thus the environment variables would be:

        GATED_TOOLS_HOMEDIR             /net/shared/working
        GATED_TOOLS_PATH                /net/shared/gated_tools
        GATED_TOOLS_USE_NAME            true

The machine trees can be created with the "create_working_dir" script. This
script expects a fourth environment variable "GATED_BUILD_DIR_NAME" to be
set to the name of the directory (not the path) where the build exists in
your shared tree. In the above example the value would be "gated". The
script will create the tree for the machine which it is run on.

You could use another symlink here call it "gated_testing" and have it point
to the build directory. Set GATED_BUILD_DIR_NAME to "gated_testing". Now you
can change which build you are testing by changing where the "gated_testing"
symlink points.

2.2 Running the Program

------------------------------------------------------------------------

To run the generator, type " testg <options>". Use the "-h" option to
generate the usage message seen below.

Usage: testg [-a] [-g] [-h] [-Q] [-v] [-t ] [-s ]

Gated test driver options:

   -a   Tells driver to use AUTO mode - input from file rather than terminal
          NOTE: this is cosmetic only: use '<' on command line to specify file
   -g   Start gated when starting driver
   -h   Print this help list
   -l   log results to /testg.log.hostname
   -M   Turn menu off
   -Q   Do not use QOS table when comparing expected results
          NOTE: this is for non-QOS versions of gated-IDRP
   -s   Set sequence and test number on startup
   -t   Set test name on startup
   -v   Set verbose mode (prints more information while running)
   -l   log results to /testg.log.hostname

Examples:

   In these examples below, the test name is "basic_12".

   1.  Run the tester in auto mode using a file for commands.
       % testg -a -t basic_12 &< commands.txt

   2.  Start gated, begin with sequence 4, test set number 1.
       % testg -g -t basic_12 -s 4 1

   3.  Run in auto mode (test name specified in input file), make me
       feel secure with lots of diagnostic messages.
       % testg -a -v < commands.txt

Note:  The "-a" option does NOT tell the tester to run from a file.  That
       is done by using the input redirector "<".  The -a option is there
       to prevent the Main Menu from printing when a file is used.

Commands may either be entered interactively (see "The Main Menu" below),
or read from an input file (see "Using a File for Commands").

The Main Menu

------------------------------------------------------------------------

1)  Select a test sequence (1t = display current test)
  Example:  1 4 1
  Sequence is specified first, then test number.  If "1" is entered
  by itself, a secondary prompt will ask for these values.
  "1t" displays the current test name, sequence, and test set number.

2)  Check gated status
  Indicates whether or not gated is running.  Returns the process ID if it is.

3)  Start gated (3c = test current config file only)
  "3c" runs gated with the "-c" flag.  This simply checks the configuration
  file to see if it parses correctly.  Routes are not actually processed.

4)  Kill gated
  Kills the gated process if one is running.

5)  Review the net status (5m = pipe to more)
  Prints the current routing table ("netstat -rn").

6)  Compare expected test results to routing table
  Runs the comparison script to compare the current expected.txt against
  the routing table and QOS file.  See the section on "Expected Results Files"
  in the configuration file generator user's guide for more information.

7)  Send refresh signal (config file re-read) to gated
  Forces a re-read of gated.conf in the working directory.

8)  Flush the routing table
  Not recommended while gated is running!

9)  Spawn a shell (csh)
  When done, simply exit the shell to return to the main menu.

10) Set the test name
   Example:  10 basic_12

11) Edit the configuration file
   Edits the current gated.conf file in the working directory.  Does not
   edit the original copy in the test directory (if it was copied from there).
   Useful for making minor changes, then sending a refresh (command #7).

12) Edit the expected results file
   Edits the current expected.txt file in the working directory.  Does not
   edit the original copy in the test directory (if it was copied from there).

13) View the gated log file
   Views (read only) the current gated-idrp log file.

14) Save the gated log and routing tables to local log files
   Uses the environment variable $gated_locallog_dir as save directory.
   Saved file uses the log file name, appending the date, sequence, and
   test set number.  (Example:  gatedlog,idrp.0917.4.1)

15) Analyze memory trace calls in the log file
   Reports on the number of dynamic memory blocks allocated and freed by
   the IDRP code.  It calculates this information by reading the MEM_TRACE
   lines in the log file.  Note: the code must have been compiled with
   -DMEM_TRACE for this feature to work.  (Really a debugging feature only.)

16) Set the time to wait between tests when in automatic mode.

21)  Turn on auto comparison - expected results/routing table
!21) Turn off auto comparison
   Example: 21 90
   Causes an automatic comparison of the routing table with the expected
   results after a test is run, waiting for the specified number of seconds
   before doing so.  This feature should be used when running commands from
   a file, to insure that gated has enough time to process route changes
   following a refresh.

22)  Turn on verbose mode (print more informational messages)
!22) Turn off verbose mode
   This causes the printing of more information/status messages, such as the
   names of working files and directories.  May provide a clue if something
   goes wrong.  And for the bored or curious, it provides some light on what
   goes on behind the scenes.

23)  Turn on synchronization with other nodes
!23) Turn off synchronization
   Example:  23 ralph norton alice
   See the section below on "Multiple Machine Synchronization."

24)  Turn on use of QOS checking.
!24) Turn off use QOS.
   Indicates whether the QOS field should be used when comparing active
   routes to the expected results file.

25) Set the working test dir.  Format is 25 <dir> <dir> <testname-dir>
   Allows you to create more hierarchy with test dirs.  The final dir is
   used to determine the test name.

26) Sync with other nodes for current test.  Format is 26 [optional_name]
   The machine will use the optional_name (if not specified the test name
   is used) to sync with the other machines given previously with the 23.

27) Kill gated and wait for it to exit.  Formate is 27 [optional_kill_arg]
   Repeatedly "kill"s gated with -TERM or the given signal name if present
   until gated exits.

28) Launch, compare, quit, flush.
   Format is 28 sequence test [ delay machine1 machine2 ... ]
   Where sequence and test select the sequence and test as in command (1).
   "delay", "machine1", etc, if given specify the machines to bring up
   under this test, the order to bring them up (left to right) and the
   delay between launching of gated on each.  Once all the machines are
   running the standard time is waited followed by comparing of the results
   killing of all the gated's and the flushing of all the routing tables.
   This allows automatic operation where each test/sequence is run in
   a completely clean environment.

!m) Turn off printing of Main Menu
m)  Turn on printing of Main Menu
   Useful if running in auto mode (thus no need to see menu), or you are
   familiar enough with the menu that you don't need to see it anymore.

0)  Exit this program
  Exits the tester.  Note: this does NOT shut down gated!!

2.3 Using a File for Commands

Commands read from an input file are specified and processed in the same
manner as if they were entered interactively.  Comment lines (lines beginning
with "#") and blank lines will be ignored.

Input files are specified using the standard input redirector "<" on the
command line as follows:
   % testg -a < <command file name<

When using a file, use of the "-a" option is highly recommended.  Otherwise,
the Main Menu will be printed after every command, as though commands were
being entered interactively.

2.4 Multiple Machine Synchronization

The tester has a feature to allow simultaneous automatic testing on multiple
machines.  Using synchronization, each machine in the group will wait for the
others to become ready before proceeding with the next refresh.

The synchronization list is specified using command:
     23 <machine> <machine> ... <machine>

This list can be modified before the start of any test, even between tests
in a sequence.

Each machine in the list uses a utility which given a string waits for
all other machines to also specify that string.  The string in the case
of changing tests using (1) or (28) is made up of the test and sequence
number.  With (26) the string is either the test name or the user supplied
string.  This allows sync'ing machines between tests.

If testing multiple configurations, machines which will not be used until
later in the test can be "sent ahead" by turning on synchronization and
setting their sequence and test set numbers.  These machines will wait until
the other machines reach that sequence and test set, and then proceed normally.

Synchronization is most useful when using the "auto comparison" mode, reading
commands from an input file.  For example, consider a test where sequence 1
uses machines "ralph" and "norton", while "alice" joins them in sequence 2.
The auto input file shown below can be used for all three machines:  assuming
no configuration files exist in sequence 1 for "alice", that machine will
skip ahead to sequence 2 and wait for the others.  When "ralph" and "norton"
reach sequence 2, "alice" will synchronize with them and continue.

# Set synchronization list
25 ralph norton

# Run sequence 1
1 1 1
1 1 2
1 1 3

# Add alice to the list
25 ralph norton alice

# Run sequence 2
1 2 1
1 2 2
1 2 3

------------------------------------------------------------------------

Author: Jeff Jackson
        Merit Network, Inc.
        jeff@merit.edu

Additional enhancements:
        Nick Amato (naamato@merit.edu)
        Shane Wright (swright@merit.edu)
        Chris Hopps (chopps@merit.edu)

$Id: testg.guide.txt,v 1.2 1997/10/21 02:28:59 naamato Exp $
