Microsoft Technology Licensing, LLC

Steve K Lim

Levent Ozgur

Timothy James Chapman

Taylor Paul Spangler

Xiaoyu Chai

Logan Ringer

Connie Qin Yang

Nithin Adapa

Matthew Scott Rosoff

Kashyap Maheshkumar Patel

Xiaoyan Liu

Aaron Edward Dietrich

Marc Mas Mezquita

Swamy V P L N Nallamalli

Alex Pung Leung

Jareth Leigh Day

The technology described herein provides an automated software-testing platform that functions in an undefined action space. The technology described herein starts with an undefined action space but begins to learn about the action space through random exploration. Both the action taken during testing and the resulting state may be communicated to a centralized testing service. The technology described herein also mines the action telemetry data and state telemetry data to identify action patterns that produce a sought after result. Once a plurality of action patterns is identified and, at least, a partial model of the action space is built, the testing on the test machines may be split into random test mode, replay test mode, and a pioneering test mode.

Automated software testing

The technology described herein provides an automated software-testing platform that uses reinforcement learning to discover how to perform tasks used in testing. The technology described herein is able to perform quality testing even when prescribed paths to completing tasks are not provided. The reinforcement-learning agent is not directly supervised to take actions in any given situation, but rather learns which sequences of actions generate the most rewards through the observed states and rewards from the environment. In the software-testing environment, the state can be user interface features and actions are interactions with user interface elements. The testing system may recognize when a sought after state is achieved by comparing a new state to a reward criteria.

Automated software testing with reinforcement learning

The technology described herein provides a cloud reinforcement-learning architecture that allows a single reinforcement-learning model to interact with multiple live software environments. The live software environments and the single reinforcement-learning model run in a distributed computing environment (e.g., cloud environment). The single reinforcement-learning model may run on a first computing device(s) with a graphical processing unit (GPU) to aid in training the single reinforcement-learning model. At a high level, the single reinforcement-learning model may receive state telemetry data from the multiple live environments. The single reinforcement-learning model selects an available action for each set of state telemetry data received and communicates the selection to appropriate the test agent. The test agent then facilitates completion of the action within the software instance being tested in the live environment. A reward is then determined for the action. The single reinforcement-learning model may then be trained.

Cloud architecture for reinforcement learning

Systems and methods directed to generating a predicted quality metric are provided. Telemetry data may be received from a from a first group of devices executing first software. A quality metric for the first software may be generated based on the first telemetry data. Telemetry data from a second group of devices may be received, where the second group of devices is different from the first group of devices. Covariates impacting the quality metric based on features included in the first telemetry data and the second telemetry data may be identified, and a coarsened exact matching process may be performed utilizing the identified covariates to generate a predicted quality metric for the first software based on the second group of devices.

Feature deployment readiness prediction

Growing community of inventors

Redmond, WA, United States of America

Steve K Lim