International Business Machines Corporation

Ping Yang

Wenbin Wang

Xiyong Han

Jin Shi

Lu Chen

Tang Xue Bo

Meng Qi Chen

Rui Na Liu

Disclosed are techniques for automated locating of user interface elements during graphical user interface testing. When a graphical user interface (GUI) is received for testing, images of the GUI are inputted to a machine learning algorithm, where image processing techniques are applied to identify groups of user interface elements and their constituent elements. Multi-dimensional index values are assigned to groups and elements corresponding to their positions within the GUI. Automated testing of the user interface elements of the GUI is performed by locating the user interface elements by their index values. If an element is not found, a scrolling technique is applied to generate an expanded virtual GUI of one or more groups of user interface elements, and the machine learning algorithm refreshes the index values using the expanded virtual GUI.

Automated locating of GUI elements during testing using multidimensional indices

Pet ramp

Pet stairs

Growing community of inventors

Ningbo, China

Ping Yang