International Business Machines Corporation

Daniel Nicolas Gisolfi

Andrew C M Hicks

Ryan Thomas Rawlins

Andrew C Hicks

Dale E Blue

Kevin Minerley

Tyler Vezio Rimaldi

Michael E Gildein

Robert Peter Catalano

Method and apparatus for regression testing. A first plurality of sequences of events is generated by executing a plurality of test paths on a first computing environment. A test workload is constructed based on the first plurality of sequences of events. A second plurality of sequences of events is generated by executing the test workload on a second computing environment. One or more variances are identified by comparing the first plurality of sequences of events with the second plurality of sequences of events. And the second computing environment is reconfigured based on the one or more variances.

Dynamic testing with captured sequences of events

A computer-implemented method includes receiving a portion of code, inputting the received portion of code to a probabilistic data structure to determine whether the portion of code is similar to one or more other portions of code, and based on determining that the one or more other portions are similar to the received portion of code, identifying a number of similar portions of code. The method also includes storing relationship data that associates the similar portions of code with the received portion of code, determining one or more subject matter experts (SMEs) associated with the similar portions of code and the received portion of code and determining correlations between SME data and the relationship data, and storing the SME data and the correlations, the SME data including an indication of the determined SMEs, the relationship data correlating the one or more SMEs to the received portion of code.

Subject matter expert identification and code analysis based on a probabilistic filter

Inputs to a system under test (SUT) are modeled as a collection of attribute-value pairs. A set of testcases is executed using an initial set of test vectors that provides complete n-wise coverage of the attribute-value pairs. For each execution of the testcases, for each attribute-value pair, a non-binary success rate (S) is computed based on the binary execution results. in response to a success rate of an attribute-value pair being below a predetermined threshold, a subset of testcases that use the attribute-value pair is identified. Further, sets of code paths for the subset of testcases are identified, each set of code path respectively corresponding to a testcase from the subset of testcases. Further, an intersection of the sets of code paths is determined. Code paths of the SUT that are in the intersection, are highlighted to represent a soft failure with the SUT.

System testing infrastructure with hidden variable, hidden attribute, and hidden value detection

A method of analyzing code is provided. The method includes generating an abstract representation of the code, identifying conditional statements in the abstract representation, populating a truth table for each conditional statement that has been identified with all possible outcomes of the conditional statement and assessing the truth table for each conditional statement to identify issues.

Using big code to construct code conditional truth tables

Aspects of the invention include determining, by a processor, a code segment of a computer code, analyzing the code segment to determine one or more other code segments associated with the code segment, determining a set of subject matter experts (SMEs) associated with the code segment and the one or more other code segments, obtaining SME data for each SME in the set of SMEs, wherein the SME data comprises a set of attributes associated with the SME, generating, by the processor, a graphical representation of the set of attributes for each SME in the set of SMEs, transforming the graphical representations into an eigenvectors and eigenvalues, and ranking the SMEs based on their associated eigenvectors and eigenvalues.

Compressing multi-attribute vector into a single eigenvalue for ranking subject matter experts

Inputs to a system under test (SUT) are modeled as a collection of attribute-value pairs. A set of testcases is executed using an initial set of test vectors that provides complete n-wise coverage of the attribute-value pairs. For each execution of the testcases, for each attribute-value pair, a non-binary success rate (S) is computed based on the binary execution results. The method further includes outputting, to a user, in response to the success rate Sof the attribute-value pair being below a predetermined threshold, an identification of one or more testcases that use the attribute-value pair, wherein the one or more testcases are to be used for diagnosing a soft failure associated with the SUT.

A system and related method comprise using a processor for executing a plurality of tests associated with a covering array of a test framework of the software test system on a first version of a system under test (SUT). For each of the plurality of tests, on a current test, the method comprises determining a current success rate value (SRV) for the current test that represents a success rate of the current test for the first version of the SUT. The method further comprises combining the current SRV of the first version of the SUT and current SRVs of the current test for prior versions of the SUT into a current test eigenvector associated with the current test. The method further comprises converting the current test eigenvector into a first eigenvalue that represents a health, accuracy, and quality of the first version of the SUT.

Verification of software test quality using hidden variables

Inputs to a system under test (SUT) are modeled as a collection of attribute-value pairs. A set of testcases is executed using an initial set of test vectors that provides complete n-wise coverage of the attribute-value pairs. For each execution of the testcases, for each attribute-value pair, a non-binary success rate (S) is computed based on the binary execution results. An attribute is selected in response to a set of success rates corresponding to a set of attribute-value pairs that includes said attribute are all below a predetermined threshold. The set of testcases is executed using another set of test vectors using additional values for the selected attribute. For each execution of the set of testcases, for each attribute-value pair, a second non-binary success rate (S') is recorded. If the predetermined threshold is now satisfied, a user is notified of the additional values for the attribute that were detected.

Inputs to a system under test (SUT) are modeled as a collection of attribute-value pairs. A set of testcases is executed using a set of test vectors that provides complete n-wise coverage of the attribute-value pairs. For each execution of the testcases, updating, for each execution of the set of testcases, for each testcase, a non-binary success rate (S) based on the binary execution results. In response to a first success rate corresponding to a particular testcase being below a predetermined threshold, a second set of testcases is generated based on the test vectors. For each testcase, a second success rate (S') is computed based on a second set of execution results of a second set of testcases. In response to the second success rate corresponding to the particular testcase being substantially same as the first success rate, a user is notified of a defect in modeling the SUT inputs.

Growing community of inventors

Hopewell Junction, NY, United States of America

Daniel Nicolas Gisolfi