Microsoft Technology Licensing, LLC

Microsoft Corporation

Michael Paul Robinson

Peter Gerber

Orville Jay Potter, Iv

Dipak Boyed

Curtis Dean Anderson

Gordon Hardy

Carlo Rivera

Jared W Wilson

Loren Merriman

Charles D McClintock

Aaron R Birnbaum

Christopher Callahan

Jing Tan

Orville Jay Potter Iv

Failure messages generated as a result of tests performed on a target application are bucketed in an effort to correlate related failure messages with minimal or no human interaction. An exemplary method for bucketing failure messages includes receiving a failure message, searching a stack trace of a target application for a blame stack trace frame from which the failure message originated, finding a most similar match bucket from a plurality of buckets, determining if the most similar match bucket at least meets a similarity threshold, adding the failure message to the most similar match bucket, if it is determined that the most similar match bucket at least meets the similarity threshold, and creating a new bucket and adding the failure message to the new bucket, if it is determined that the most similar match bucket does not at least meet the similarity threshold.

Test failure bucketing

Detecting machine utilization activity may be provided. Client machine activity within a computer network may be monitored to determine if the client machine is available for use, or if the computer is already engaged in one or more activities. During a predetermined receive period, machine activity information for a given client machine. The machine activity information for the client machine is compared to previously stored machine activity information to determine if an activity state for the client machine has changed. If the activity state of the client machine has changed, the activity state for the machine is updated. Machine activity information for all computers in the computer network may be used to track and optimize resources with the computer network.

Detecting machine utilization activity

Break and optional hold preserves a state of a computing environment on which a software program has failed. Being able to examine the status of the environment existing upon the occurrence of the failure, including the condition of various processes and values facilitates resolution of the cause of the failure. Upon occurrence of a failure during the execution of a first software program in a first computing environment, execution of the first software program breaks. A first state of the first computing environment existing upon the breaking in execution of the first software program is then held. A failure notification is generated to signal the failure to a monitoring system. The monitoring system accesses hold information to determine whether the first computing environment should hold its current state and whether one or more other computing environments interacting with the first computing environment should also hold their states.

Break and optional hold on failure

An integrated test framework is disclosed that allows software testers to easily generate and execute tests of software involving multiple interacting computer systems. A copy of the integrated test framework resides on each computer system in the test. The integrated test framework on each computer system supports the independent testing of software on that system and also the synchronization between the computer systems. A test manager is provided to coordinate the synchronization. All the information necessary to direct the test framework on each of the computer systems is included within a single test script that is propagated to and executed on each computer system. The test script dictates the role of each computer system within the test and includes role specific actions and identifies points at which the computer systems must synchronize together for the passing of data, messages or other communications.

Multi-machine testing system and method for testing software

A test services provider is an object that is arranged to provide a standardized interface for calling test harness provided functionality. The test services provider is not tied to a specific test harness, and provides a standard interface such that both test harness functionality and test cases can be written without beforehand knowledge of the other. In operation, the test harness registers the test services that provides with the test service provider, and the test cases retrieve the registered test services from the test harness as needed. Accordingly, the test case does not have to query the test harness itself for the object that provides the desired test harness functionality, but rather uses a standardized interface provided by the test services provider to retrieve the desired test services.

Test services provider

The present invention is directed towards a test case inheritance behavior that can be controlled via attributes. A base test class from which test objects are derived are useful for reducing test case code and management. For example, base test classes and their derived objects can be used to implement steps that are common between the entire set of classes (e.g., launching a piece of software to be tested and getting it to a certain stage). The principle of inheritance simplifies management of the test software when, for example, the base class is modified, all of the tests which derive from that test case automatically are modified. Accordingly, only one item needs to be modified (instead of every tests) when a change is necessary to modify the way the software launches.

Test case inheritance controlled via attributes

The integrated test framework of the present invention includes a test framework runtime that executes test scripts and that also allows a script to identify a variation tool called a 'data provider' that is responsible for providing data to the test framework runtime and controlling the iteration of the runtime through the variations. The script also identifies the type of data set from which the data provider should obtain the data from which the variations are derived and the location of the data set. Multiple variation data providers may be used in conjunction with an adapter layer that coordinates the iteration of each variation data provider and creates an aggregated variation over multiple varying parameters of different scope.

System and method for testing software using data-driven test variations

Smart test attributes are objects that are arranged to incorporate attributes functionality in the attributes themselves and to provide a standard way to interact with the attributes through a set of interfaces. The set of interfaces includes interfaces for class level attributes, method level attributes, and parameter level attributes. Test class attributes can be used to modify an instantiated object state, skip test extraction from a class, or cause test extraction to be performed multiple times for a denoted type. Method level attributes are capable of modifying method parameters and method execution. In an embodiment, method level attributes include execution attributes and supplemental attributes. Parameter level attributes are optionally used to modify the parameter input to a method and/or to modify a state of an object (e.g., the context) after a method has executed.

Smart test attributes and test case scenario in object oriented programming environment

A test method executor is an object designed to execute a test method. The test method executor typically implements a simple state engine that is used to process the data presented by each smart test attribute. The test method executor invokes particular test methods in response to the processed data. The test method executor then condenses the results from all the attributes typically into a simple return value, which is designed to be interpreted by the test harness. The return value indicates, for example, whether the executed test method was a success, failure, or if the test method was 'skipped.' Additionally, the test method executor is used to gather data about the method being invoked (such as the method name, which attributes are present, and the like.)

Growing community of inventors

Redmond, WA, United States of America

Michael Paul Robinson