Kofax, Inc.

Tungsten Automation Corporation

Kapow Technologies, Inc.

Vadim Alexeev

Benjamin De Coninck Owe

The presently disclosed inventive concepts are directed to systems, computer program products, and methods for intelligent screen automation. According to one embodiment, a method includes: determining one or more logical relationships between textual elements and non-textual elements of one or more images of a user interface; building a hierarchy comprising some or all of the non-textual elements and some or all of the textual elements in order to form a data structure representing functionality of the user interface; and outputting the data structure to a memory.

Determining functional and descriptive elements of application images for intelligent screen automation

The presently disclosed inventive concepts are directed to systems, computer program products, and methods for intelligent screen automation. The inventive computer program products include program instructions configured to cause a computer, upon execution thereof, to perform a method including: identifying, from among a plurality of elements within one or more images of a user interface, a first set of elements and a second set of elements, wherein the first set of elements and the second set of elements each independently comprise either or both of: textual elements, and non-textual elements; determining one or more logical relationships between the textual elements and the non-textual elements; building a hierarchy comprising some or all of the first set of elements and some or all of the second set of elements, wherein building the hierarchy forms a data structure representing functionality of the user interface; and outputting the data structure to a memory.

The presently disclosed inventive concepts are directed to systems, computer program products, and methods for intelligent screen automation. The inventive techniques include: identifying first and second sets of elements within one or more images of a user interface, where each of the first set of elements is independently selected from: vertical lines, horizontal lines, and rectangular elements, and each of the second set of elements is independently selected from: radio buttons, icons, and textual elements. The methods also include determining one or more logical relationships between the textual elements and non-textual elements of the image; building a hierarchy comprising some or all of the first set of elements and some or all of the second set of elements in order to form a tree-based data structure representing functionality of the user interface; and outputting the tree-based data structure to a memory.

Growing community of inventors

Saint-Petersburg, Russia

Vadim Alexeev