International Business Machines Corporation

Paul Regis Kram

Kenneth Nagin

Alan Hartman

Eitan D Farchi

Shmuel Ur

Daniel Geist

Yael Shaham-Gafni

A test coverage tool provides output that identifies differences between the actual coverage provided by a test suite run on a program under test and the coverage criteria (e.g., the coverage criteria required by the test/development team management). The output from the test coverage tool is generated in the same language that was used to write the coverage criteria that are input to an automated test generator to create the test cases which form the test suite. As a result, the output from the coverage tool can be input back into the automated test generator to cause the generator to revise the test cases to correct the inadequacies. This allows iterative refinement of the test suite automatically, enabling automated test generation to be more effectively and efficiently used with more complex software and more complex test generation inputs. In preferred embodiments, test coverage analysis results of several different test suites, some manually generated and others automatically generated, are used to produce a streamlined automatically-generated test suite and/or to add missing elements to an automatically generated test-suite.

Method and system for integrating test coverage measurements with model based test generation

The present invention enables the modeling of plural outcomes resulting from a single stimulus, thereby allowing for automated test generation for non-deterministic software. In accordance with the present invention, a method, system, and computer program product are disclosed for testing software having a non-deterministic outcome. In accordance with the present invention, a set of rules is created, each rule having an associated stimulus, a precondition computation, and a computation procedure to produce the set of all valid outcome states. Each valid outcome state models one of several states that may result from applying the stimulus to any state which satisfies the precondition computation. Using these models, a test generation program is executed against the set of rules to recursively expand each path associated with an outcome state of a rule and outputting a list of valid possible outcomes, then selecting sequences of stimuli in accordance with coverage goals set by the test engineer. A test execution engine is then run against the application under test applying the sequences of stimuli to the software and a list of observed states resulting from the application of the stimuli is compiled. Finally, the list of observed states is compared with the list of valid possible outcome states to identify defects (as well as to identify correct outcomes) in the application under test and ensure the attainment of the coverage goals.

Method, system, and computer program product for automated test generation for non-deterministic software using state transition rules

A system for automatic generation of test programs employs test generation foci linked to a finite state machine behavioral model of a software application under test to produce abstract test suites that are executed by an execution engine. The foci include directive expressions that tag coverage variables of the behavioral model. The execution engine operates until the tagged coverage variables have assumed all their possible values. Both the behavioral models and the foci are archivable and can be retrieved and reused independently.

Technique using persistent foci for finite state machine based software test generation

A method and system for emulating network latency, packet corruption, packet shuffling, packet loss and network congestion is introduced so that network connected multi-computer software systems can be tested and debugged in a cost effective and efficient manner. This network emulator requires no changes to the software being tested and requires only modifications to the MAC to IP mapping tables of the computers running the software to be tested. IT requires no modification to the communication stacks of the computers involved. The changes to these tables cause packets to be redirected to an emulator host computer where they can be delayed, deleted, corrupted or shuffled prior to delivery to their final destination.

Growing community of inventors

Pittsburgh, PA, United States of America

Paul Regis Kram