Micro Focus Lcc

Shuhui Fu

Er-Xin Shang

Peng-Ji Yin

Yun-Sheng Liu

Yun Yuan

Yi-Bin Guo

Hua-Ming Zhai

Amos Nesher

Anton Kaminsky

Gil Nakache

Eyal Luzon

Tsachi Ben Zur

Xiao-Fei Yu

An image of a graphical user interface is captured. A process is started to record identification of a graphical object (e.g., a text field) in the image of the graphical user interface. Recording identification of the graphical object in the image of the graphical user interface is based on machine learning. An end to the process to record identification of the graphical object in the image of the graphical user interface is identified. A machine learned software object is updated. For example, the machine learned software object is updated with a location of the graphical object. The updated machine learned software object then is used to test the graphical object. For example, to automatically fill in the text field using a test script.

Machine learning based on functional testing

Applications under test (AUT) may be tested by automated testing systems utilizing machine vision to recognize visual elements presented by the AUT and apply inputs to graphical elements, just as a human would. By utilizing the smallest image patch available, processing demands of the testing system are minimized. However, the image patch used to identify a portion of an AUT must be identifiable to the automated system. By selecting image patches that comprise the smallest size, but can be identified in an AUT by an automated system using machine vision, even as the AUT display is resized, reproportioned, noisy, or otherwise altered from the testing platform that was utilized for training.

Automated application testing of mutable interfaces

Embodiments provide for automated testing of an Application Under Test (AUT) that utilizes an infinite scroll element. Automated testing of the AUT can comprise executing one or more test scripts on the AUT which can present a user interface including a plurality of elements and the test scripts can perform one or more functional tests on the AUT through the plurality of elements of the user interface. During performance of the one or more functional tests, a scroll event associated with an element of the user interface can be detected and a determination can be made as to whether the element associated with the scroll event is an infinite scroll element. In response to determining the element is an infinite scroll element, a set of test results for the functional test can be recorded including results for the infinite scroll element and an interaction with the infinite scroll element.

Supporting record and replay for infinite scroll elements

An image of a user interface of a device is captured. A graphical object is identified in the image of the user interface. For example, a menu item or a text object is identified in the image of the user interface. An Artificial Intelligence (AI) process is used to determine a confidence score for the graphical object; the confidence score identifies a confidence of how likely a type of the graphical object can be identified. In response to the first confidence score not meeting a threshold, a zoomed in image of the graphical object is taken or a zoomed-out image of the graphical object is taken. The zoomed in image or the zoomed-out image is used to increase the confidence score to better identify the type of the graphical object.

Automatic zoom on device screen to improve artificial intelligence identification rate

Growing community of inventors

Shanghai, China

Shuhui Fu