Method and apparatus for environment-variable driven software testing

Abstract

An apparatus and method for testing software is disclosed. A test base is constructed on a storage device by creating a plurality of directories connected to form a hierarchical directory structure. Files associated with software programs to be tested are stored in directories within the hierarchical directory structure. Such files may include source code files, test input files and expected output files. Environment files and environment configuration files that specify values for environment variables are also stored in the test base. A test is run by invoking a test engine and passing to the test engine the name of an environment file and the name of a test. The test engine modifies the values of environment variables based on the specified environment file prior to executing the specified test. After modifying the environment based on the specified environment file and prior to executing the test, the test engine further modifies the environment based on any applicable environment configuration files. The test engine determines which environment configuration files are applicable to a test based on the location of the environment configuration files in the test base relative to the location of the directory that contains the files associated with the test. In addition to single test operations, the test engine may perform a series of tests based on a test list. The test apparatus also includes a tool for storing the results of past tests, and a tool for generating test lists in which tests are ordered based on the results of past tests. Multiple tests are performed simultaneously by distributing the test processes among a plurality of processors.