T-mobile Usa, Inc.

Dong Chen

Tor Fredericks

Pei Zheng

Anqi Luo

Described herein are techniques that may be used to automate testing of services on mobile devices using visual analysis. In some embodiments, a machine learning model is trained using interaction data received from a number of mobile devices by correlating user selections with visual elements (e.g., icons). During execution of a testing routine on a mobile device, screenshots are obtained of a screen of the mobile device and provided to the machine learning model. An action is generated based on the provided screenshot that simulates a user action (e.g., a user touch on the screen of the mobile device) at a location of an icon or other visual element associated with the testing routine. These steps are repeated until an end-state of the testing routine is detected.

Automated testing of mobile devices using visual analysis

Described herein are techniques that may be used to automate testing of services on mobile devices using visual analysis. In some embodiments, a behavioral model or other machine learning model is trained using training data collected while testers use mobile devices to test the services. During execution of a testing routine on a mobile device, screenshots are obtained of a screen of the mobile device and provided to the machine learning model. The behavioral model or other machine learning model can use the provided screenshot to determine an action that simulates a user action (e.g., a user touch on the screen of the mobile device) at a location of an icon or other visual element associated with the testing routine. These steps are repeated until an end-state of the testing routine is detected.

Growing community of inventors

Lansing, MI, United States of America

Dong Chen