International Business Machines Corporation

Deborah A Furman

Anthony Thomas Sofia

Andrew C M Hicks

Dale Francis Riedy, Jr

Ryan Thomas Rawlins

Matthew J Kalos

Harry M Yudenfriend

Beth Ann Peterson

David Michael Shackelford

Susan Kay Candelaria

Scott Brady Compton

John Richard Paveza

Ilene A Goldman

Michael E Gildein

Marco Kraemer

Andrew C Hicks

Dale E Blue

Michael Terrence Cohoon

Rachel Brill

Heather M Bosko

Bradley M Messer

Andrew Cm Hicks

Aspects of the invention include mutating each neural network of a portion of a first array of neural networks, wherein each neural network of the first array of neural networks is configured to select a respective sequence of test cases for testing a computing infrastructure. Causing each neural network of a second array of neural networks to select a respective sequence of test cases for testing the computing infrastructure. Generating a child neural network by performing a crossover operation between a mutated neural network of the portion of the first array and a neural network of the second array of neural networks, the child neural network generating a new sequence of test cases for testing the computing infrastructure.

Reinforcement learning for testing suite generation

Aspects of the invention include creating a neural network including neurons to which actions are assigned in representation of test cases. Tests of various instantiations of the neural network are executed for each test case and a state of the neural network after each test is evaluated to determine a fitness score of a corresponding instantiation. Instantiations having fitness scores that exceed a predefined level and identified and the instantiations having the fitness scores that exceed the predefined level are selected for adjustments. The executing, the evaluating, the identifying and the selecting are iteratively repeated in order to obtain desired fitness scores.

Reinforcement testing of a neural network

Techniques for runtime metrics based test ordering in a computer system are described herein. An aspect includes determining a first runtime metric associated with a first module based on the execution of a first test case. Another aspect includes determining a second runtime metric associated with a second module based on the execution of a second test case. Another aspect includes comparing the first runtime metric and the second runtime metric. Another aspect includes determining an order of a plurality of test cases based on the comparison of the first runtime metric and the second runtime metric. Another aspect includes executing the plurality of test cases in the determined order.

Runtime metrics based test ordering

A method for detecting and localizing a fault in a system under test (SUT) includes generating an initial set of test vectors that provides complete n-wise coverage of the reduced test space. The method further includes generating and executing an initial set of test cases to obtain a first set of execution results. The method further includes determining, based at least in part on the first set of execution results, that one or more test cases failed execution. The method further includes generating a set of new test cases from a selected failing test case. The method further includes executing the set of new test cases to obtain a second set of execution results. The method further includes detecting and localizing the fault based at least in part on the second set of execution results.

Executing tests in deterministic order

Aspects of the invention include determining a functional coverage model for a system under test (SUT), the functional coverage model comprising a plurality of attributes, wherein each attribute includes a set of values, determining a set of test cases having n-wise test coverage for the SUT, executing one or more test cases to determine an execution bottleneck value in a first set of values for a first attribute, analyzing the set of test cases to determine a first test case having the execution bottleneck value for the first attribute, the execution bottleneck value having a larger execution metric than a second value, determining that the first set of values for the first attribute in the first test case is moot for the n-wise test coverage for the SUT, and replacing the execution bottleneck value for the first attribute in the first test case with the second value.

Optimizing test case execution

A computer-implemented method for testing a system under test (SUT) in an active environment includes generating multiple test cases. The method further includes executing, by the testing system, a first set of test cases from those generated, and determining a test case that failed. Further a code path that is exercised by the test case is detected, the code path having a defect. The method further includes identifying, by the testing system, a second set of test cases from the test cases that are generated, wherein each test case from the second set of test cases has not been executed and exercises the code path that is identified. The method further includes reordering, by the testing system, at runtime, the test cases to omit execution of the second set of test cases.

Dynamic reordering of test case execution

A method for automatically avoiding fault paths in software code of a System Under Test (SUT) includes generating a plurality of fingerprints by executing a plurality of regression tests. Each of the plurality of fingerprints uniquely identifies a specific code path in the software code of the SUT. A critical error is detected during execution of the software code of the SUT. A fault code path in the software code of the SUT associated with the critical error is identified by analyzing the plurality of generated fingerprints. At least one fingerprint associated with the fault code path in the software code of the SUT is identified. During subsequent execution of the software code of the SUT, the identified fault code path in the software code of the SUT is automatically prevented from being executed based on the identified at least one fingerprint.

Self healing software utilizing regression test fingerprints

A method for detecting and localizing a fault in a system under test (SUT) includes generating an initial set of test vectors that provides complete n-wise coverage of the reduced test space. The method further includes generating an initial set of test cases from the initial set of test vectors and executing the initial set of test cases. The method further includes generating a set of new test cases from a selected failing test case, wherein generating the set of new test cases comprises generating, in relation to each attribute in the selected failing test case, a respective subset of new test cases at least in part by changing a respective attribute value for the attribute in the selected failing test case to each other candidate attribute value for the attribute that is not present in any of the one or more test cases that failed execution.

Reduction of testing space for system testing infrastructure using combinatorics

Embodiments include accelerating capture of a system dump at system failure in a computer system. A method includes capturing a system snapshot that includes data from real memory frames of the computer system and storing the system snapshot in a storage. The method also includes monitoring periodically a change between the data in the system snapshot and a current state of the real memory frames of the computer system and updating the system snapshot according to the changed real memory frames based on a rate of change of the real memory frames. Further, in response to encountering the system failure, the snapshot is updated based on a delta between the system snapshot and the current state of the real memory frames. The snapshot on the storage is used as the system dump. The present document further describes examples of other aspects such as systems, computer products.

Accelerating system dump capturing

A computer program product for optimizing page table manipulations is provided and includes a computer readable storage medium having program instructions that are readable and executable by a processing circuit to cause the processing circuit to create and maintain a translation table with a translation look-aside buffer (TLB) disposed to cache priority translations, update the translation table upon de-registration of a DMA address, allocate entries in the translation table from low to high memory addresses during memory registration, maintain a cursor for identifying where to search for available entries upon performance of a new registration, advance the cursor from entry-to-entry in the translation table and wrap the cursor from an end of the translation table to a beginning of the translation table and issue a synchronous TLB invalidation instruction to invalidate the TLB upon at least one wrapping and an entry being identified and updated.

Optimizing page table manipulations

Embodiments of the present invention provide systems and methods for generating a set of test cases using a base test program. The base test program may be used as both a functional drive and as a performance measuring test case. From the base test program, additional key and value pairs may be added to the base test program to force specific test scenarios.

Automated dynamic test case generation

Embodiments include methods, and adaptive testing systems, and computer program products for performing adaptive testing using one or more system resources of a computer system dynamically determined from a platform on which a test program is executing. Aspects include: test program sending a resource query based on certain criteria to a resource query module to inquire one or more available system resources of computer system, the resource query module using certain operating system commands of computer system to determine appropriate system resources available for use on the computer system, the computer system returning the appropriate system resources determined on the computer system to the resource query module of the test program, the test program deciding one or more system resources that best meet a need of the test program, and the test program performing the adaptive testing on the computer system based on the one or more system resources decided.

Growing community of inventors

Staatsburg, NY, United States of America

Deborah A Furman