Automation Anywhere, Inc.

Intel Corporation

Haileo, Inc.

Sudhir Kumar Singh

Virinchipuram J Anand

Vinod Sharma

Malini Krishnan Bhandaru

Monica Lucia Martinez-Canales

Vwani P Roychowdhury

Jesse Truscott

Harshil Lodhiya

Nima Sarshar

Jose M Rodriguez

Prasad Modali

Abhishek Narain

Robotic process automation (RPA) systems with improved playback capabilities. Certain embodiments can provide resilient playback of software automation processes by providing the capability to record, compute and store parameters for user interface controls detected from a screen image of a user interface. These parameters can be used to assist in locating correct corresponding user interface controls within a screen image presented at playback of a software automation process. Other embodiments can, additionally or alternatively, provide resilient playback of software automation processes by providing enhanced capability to locate user interface controls within a screen image of a user interface. In some embodiments, one or more of the user interface controls located within the screen image of the user interface can be used to manipulate the user interface so that other user interface controls become visible within the screen image. Advantageously, embodiments disclosed herein allow software automation processes to operate with greater reliability and flexibility.

Robotic process automation using enhanced object detection to provide resilient playback capabilities

Robotic process automation (RPA) systems with improved playback capabilities are disclosed. Certain embodiments can provide resilient playback of software automation processes by providing enhanced capabilities to locate user interface controls within a screen image of a user interface of an application program being utilized by the software automation processes. Using such capabilities, embodiments can provide resilient playback of software automation processes by providing the capability to resolve playback errors by detecting user interface controls within at least one screen image of a user interface that was captured when the software automation process was created. Advantageously, embodiments disclosed herein allow software automation processes to continue to operate over time and thus with greater resiliency and flexibility.

Robotic process automation with resilient playback of recordings

Computerized detection of one or more user interface objects is performed by processing an image file containing one or more user interface objects of a user interface generated by an application program. Sub-control objects can be detected in the image file, where each sub-control object can form a portion of a user interface object that receives user input. Extraneous sub-control objects can be detected. Sub-control objects that overlap with or that are within a predetermined vicinity of an identified set of sub-control objects can be removed. Sub-control objects in the identified set of sub-control objects can be correlated to combine one or more of the sub-control objects in the identified set of sub-control objects to generate control objects that correspond to certain of the user interface objects of the user interface generated by the application program.

Machine learned retraining for detection of user interface controls via variance parameters

Robotic process automation (RPA) systems with improved playback capabilities. Certain embodiments can provide resilient playback of software automation processes by providing enhanced capability to locate user interface controls within a screen image of a user interface. In some embodiments, one or more of the user interface controls located within the screen image of the user interface can be used to manipulate the user interface so that other user interface controls become visible within the screen image. Other embodiments can, additionally or alternatively, provide resilient playback of software automation processes by providing the capability to record, compute and store parameters for user interface controls detected from a screen image of a user interface. These parameters can be used to assist in locating correct corresponding user interface controls within a screen image presented at playback of a software automation process. Advantageously, embodiments disclosed herein allow software automation processes to operate with greater reliability and flexibility.

Robotic process automation with resilient playback capabilities

Detection of user interface controls via invariance guided sub-control learning

A quaternion deep neural network (QTDNN) includes a plurality of modular hidden layers, each comprising a set of QT computation sublayers, including a quaternion (QT) general matrix multiplication sublayer, a QT non-linear activations sublayer, and a QT sampling sublayer arranged along a forward signal propagation path. Each QT computation sublayer of the set has a plurality of QT computation engines. In each modular hidden layer, a steering sublayer precedes each of the QT computation sublayers along the forward signal propagation path. The steering sublayer directs a forward-propagating quaternion-valued signal to a selected at least one QT computation engine of a next QT computation subsequent sublayer.

Computationally-efficient quaternion-based machine-learning system

A machine-learning system includes a quaternion (QT) computation engine. Input data to the QT computation engine includes quaternion values, each comprising a real component and three imaginary components, represented as a set of real-valued tensors. A single quaternion value is represented as a 1-dimensional real-valued tensor having four real-valued components, wherein a first real-valued component represents the real component of the single quaternion value, and wherein a second, a third, and a fourth real-valued component each respectively represents one of the imaginary components. A quaternion-valued vector having a size N is represented as a 2-dimensional real-valued tensor comprising N 1-dimensional real-valued tensors. A quaternion-valued matrix having N&#xd7;M dimensions is represented as a 3-dimensional real-valued tensor comprising M 2-dimensional real-valued tensors comprising N 1-dimensional real-valued tensors.

Tensor-based computing system for quaternion operations

A User Interface (UI) interface object detection system employs an initial dataset comprising a set of images, that may include synthesized images, to train a Machine Learning (ML) engine to generate an initial trained model. A data point generator is employed to generate an updated synthesized image set which is used to further train the ML engine. The data point generator may employ images generated by an application program as a reference by which to generate the updated synthesized image set. The images generated by the application program may be tagged in advance. Alternatively, or in addition, the images generated by the application program may be captured dynamically by a user using the application program.

Learning user interface controls via incremental data synthesis

A deep neural network (DNN) includes hidden layers arranged along a forward propagation path between an input layer and an output layer. The input layer accepts training data comprising quaternion values, outputs a quaternion-valued signal along the forward path to at least one of the hidden layers. At least some of the hidden layers include quaternion layers to execute consistent quaternion (QT) forward operations based on one or more variable parameters. A loss function engine produces a loss function representing an error between the DNN result and an expected result. QT backpropagation-based training operations include computing layer-wise QT partial derivatives, consistent with an orthogonal basis of quaternion space, of the loss function with respect to a QT conjugate of the one or more variable parameters and of respective inputs to the quaternion layers.

Gradient-based training engine for quaternion-based machine-learning systems

Computerized detection of one or more user interface objects is performed by processing an image file containing one or more user interface objects of a user interface generated by an application program. Sub-control objects are detected in the image file, where each sub-control object forms a portion of a user interface object that receives user input. Extraneous sub-control objects are detected. Sub-control objects that overlap with or that are within a predetermined vicinity of an identified set of sub-control objects are removed. Sub-control objects in the identified set of sub-control objects are correlated to combine one or more of the sub-control objects in the identified set of sub-control objects to generate control objects that correspond to certain of the user interface objects of the user interface generated by the application program.

A method of emotive autography includes calculating a plurality of classifiers associated with an individual user. Each of the classifiers indicates a preference of the user for an associated type of multimedia content. Multimedia data is received including video data, audio data and/or image data. The multimedia data is divided into semantically similar segments. A respective preference score is assigned to each of the semantically similar segments by use of the classifiers. The semantically similar segments are arranged in a sequential order dependent upon the preference scores. An emotive autograph is presented based on the semantically similar segments arranged in the sequential order.

System architecture for universal emotive autography

This invention presents a universal framework for the discovery, understanding and matching of dress styles. In one embodiment, a computer-implemented method for building a universal dress style learner is disclosed, said method comprising: learning human skin models; detecting skin using the learned human skin models; collecting a set of dress images worn by a model; computing a set of style features based on the skin detected for at least one subset within the set of dress images; computing a set of clusters on the at least one subset of dress images based on at least one subset of the set of style features; validating the set of clusters for the at least one subset of style features; and computing a set of validated style features and a style basis.

Methods and systems for building a universal dress style learner

A unified framework to understand multimedia signals utilizes the loosely annotated multimedia data on the Web, analysis it in various signal domains, such as text, image, audio and combinations thereof, and builds an association graph called the 'Multimedia Brain,' which basically comprises visual signals, audio signals, text phrases and the like that capture a multitude of objects, experiences and their attributes and the links among them that capture similar intent or functional and contextual relationships.

Systems and methods for building a universal multimedia learner

'A Universal Learner (UL)' provides a unified framework to understand multimedia signals. The UL utilizes the loosely annotated multimedia data on the Web, analyses it in various signal domains, such as text, image, audio and combinations thereof, and builds an association graph called the 'Multimedia Brain,' which basically comprises visual signals, audio signals, text phrases and the like that capture a multitude of objects, experiences and their attributes and the links among them that capture similar intent or functional and contextual relationships.

Growing community of inventors

Dublin, CA, United States of America

Sudhir Kumar Singh