Automation Anywhere, Inc.

Jesse Truscott

Virinchipuram J Anand

Sudhir Kumar Singh

Harshil Lodhiya

Robotic process automation (RPA) systems with improved playback capabilities. Certain embodiments can provide resilient playback of software automation processes by providing the capability to record, compute and store parameters for user interface controls detected from a screen image of a user interface. These parameters can be used to assist in locating correct corresponding user interface controls within a screen image presented at playback of a software automation process. Other embodiments can, additionally or alternatively, provide resilient playback of software automation processes by providing enhanced capability to locate user interface controls within a screen image of a user interface. In some embodiments, one or more of the user interface controls located within the screen image of the user interface can be used to manipulate the user interface so that other user interface controls become visible within the screen image. Advantageously, embodiments disclosed herein allow software automation processes to operate with greater reliability and flexibility.

Robotic process automation using enhanced object detection to provide resilient playback capabilities

Robotic process automation (RPA) systems with improved playback capabilities. Certain embodiments can provide resilient playback of software automation processes by providing enhanced capability to locate user interface controls within a screen image of a user interface. In some embodiments, one or more of the user interface controls located within the screen image of the user interface can be used to manipulate the user interface so that other user interface controls become visible within the screen image. Other embodiments can, additionally or alternatively, provide resilient playback of software automation processes by providing the capability to record, compute and store parameters for user interface controls detected from a screen image of a user interface. These parameters can be used to assist in locating correct corresponding user interface controls within a screen image presented at playback of a software automation process. Advantageously, embodiments disclosed herein allow software automation processes to operate with greater reliability and flexibility.

Robotic process automation with resilient playback capabilities

A User Interface (UI) interface object detection system employs an initial dataset comprising a set of images, that may include synthesized images, to train a Machine Learning (ML) engine to generate an initial trained model. A data point generator is employed to generate an updated synthesized image set which is used to further train the ML engine. The data point generator may employ images generated by an application program as a reference by which to generate the updated synthesized image set. The images generated by the application program may be tagged in advance. Alternatively, or in addition, the images generated by the application program may be captured dynamically by a user using the application program.

Growing community of inventors

Petaluma, CA, United States of America

Jesse Truscott