International Business Machines Corporation

Michele Dolfi

Peter Willem Jan Staar

Christoph Auer

Konstantinos Bekas

Leonidas Georgopoulos

Birgit Monika Pfitzmann

Aleksandros Sobczyk

Alexander Velizhev

Ralf Kaestner

Kasper Dinkla

Gerhard Ingmar Meijer

Martin M Rufli

Ahmed Samy Nassar

Elliot Jacques Vincent

Lokesh Mishra

André Carvalho

Peter Staar

Tim Jan Baccaert

Dal Noguer Hidalgo

Rita Kuznetsova

In an approach for a dynamic in-memory construction of a knowledge graph structure, the knowledge graph structure comprising a plurality of nodes and edges linking selected nodes to each other, a processor receives a record comprising a plurality of strings. The plurality of strings relates to a command combined with a set of strings. A processor determines content records relating to nodes relating to each of the strings. A processor assigns node identifiers for respective determined content records. A processor appends the node identifiers to a dynamic in-memory knowledge graph structure. A processor modifies an edge between selected ones of the node identifiers based on the command combined with the set of strings. A processor builds the dynamic in-memory knowledge graph structure.

Dynamic in-memory construction of a knowledge graph

The present disclosure relates to a method for searching a graph representing content of digital objects. A set of operations for traversing the graph may be determined according to a search request. The set of operations may be executed, resulting in intermediate result vectors of nodes and a result vector of nodes, wherein the result vector of nodes is associated with a result set of one or more object units of the digital objects. Intermediate result vectors may be selected from of the intermediate result vectors. A set of result entities may be identified. The set of result entities are entities which are part of the object units and part of entities represented by nodes of said selected intermediate result vectors. The set of result entities and the result set of object units may be provided as a result of the search request.

Search in knowledge graphs

Provided are techniques for performing a search using a hypergraph. Entities are identified. A knowledge graph using the entities is generated, wherein nodes of the knowledge graph represent the entities and edges between the nodes represent pair-wise relationships, and wherein each of the edges carries an edge score that quantifies a degree of coherence between a pair of the entities. A hypergraph using the knowledge graph is generated, wherein nodes of the hypergraph represent the entities and hyperedges represent relationships between multiple entities, and wherein each of the hyperedges carries a hyperedge score that quantifies a degree of coherence between the multiple entities. A search request is received. A search result is generated using the hypergraph, wherein the search result comprises a set of coherently related entities. The search result is returned.

Performing a search using a hypergraph

The present disclosure relates to a method for displaying a search result in a user interface comprising a sequence of K table rows, a sequence of first type columns (named regular columns) and a sequence of B second type columns (named braided columns), the search result comprising values of attributes. The method includes creating a structure that includes a sequence table rows and a sequence of regular columns and a sequence of braided columns. The method selects a row of the sequence of rows and a braided column of the braided columns for displaying the search result. In each regular column a regular cell that is included in the selected row may be defined. From a braided column a braided cell may be defined that includes in the selected row and subsequent B−1 rows. The attributes values are displayed in the defined regular and braided cells.

Presentation of search results in a user interface

The present disclosure relates to a method for generating a structure of a PDF-document, wherein the PDF-document comprises elements. The method comprises detecting document cells of the PDF-document dependent on commands of a page description language for printing the elements of the PDF-document. The method comprises determining parts of the PDF-document dependent on the PDF-document by a machine learning module. The determining of the respective part comprises associating a respective portion of the elements of the PDF-document with the respective part. Furthermore, a respective label may be assigned to the respective part. The method may further comprise using a symbolic artificial intelligence module, wherein rules of the symbolic AI-module for reconciling the document cells with the parts may be applied. The elements of the structure of the PDF-document may be generated and labelled dependent on a result of the reconciling and dependent on the respective label to the respective part.

Generating a structure of a PDF-document

A computer-implemented method for determining a set of target items to be annotated for training a machine learning application. The method comprises providing a training data set with a set of data samples and an auto-encoder with a classifier. The auto-encoder comprises an embedding model that maps the set of data samples to a set of compressed feature vectors. The set of compressed feature vectors define a compressed feature matrix. Further provided are: a definition of a graph associated to the compressed feature matrix, applying a clustering-algorithm to identify node clusters of the graph and applying a centrality algorithm to identify central nodes of the node clusters, retrieving from an annotator node labels for the central nodes, propagating the annotated node labels to other nodes of the graph and performing a training of the embedding model and the classifier with the annotated and the propagated node labels.

Efficient ground truth annotation

A computer-implemented method for processing a digital image. The digital image comprises one or more text cells, wherein each of the one or more text cells comprises a string and a bounding box. The method comprises receiving the digital image in a first format, the first format providing access to the strings and the bounding boxes of the one more text cells. The methods further comprises encoding the strings of the one or more text cells as visual pattern according to a predefined string encoding scheme and providing the digital image in a second format. The second format comprises the visual pattern of the strings of the one or more text cells. A corresponding system and a related computer program product is provided.

Digital image processing

A method, system and computer program product to generate a training data set for image segmentation applications, comprising providing a set of input documents of a first format. The input documents each comprise one or more pages. The input documents are split into individual document pages and parsed. Parsing comprises identifying a predefined set of items including position information of the position of the predefined set of items in the individual document pages; generating a bitmap image of a second format for each individual document page of the first format. The bitmap image comprises a predefined number of pixels. A mask is generated for each individual document. The mask comprises the predefined number of pixels of the corresponding bitmap image. Generating the mask comprises assigning an encoded class label to each pixel of the mask based on the position information of identified items of the predefined set of items.

Ground truth generation for image segmentation

Aspects of the present invention disclose a method, computer program product, and system for performing a multiplication of a matrix with an input vector. The method includes one or more processors subdividing a matrix into logical segments, the matrix being given in a sparse-matrix data format. The method further includes one or more processors obtaining one or more test vectors. The method further includes one or more processors performing an optimization cycle. In an additional aspect, performing the optimization cycle further comprises, for each of the test vectors, one or more processors, performing a cache performance test.

Multiplication of a matrix with an input vector

The invention is notably directed to a computer-implemented method for processing a plurality of documents. The method comprises providing the plurality of documents in a first format and splitting each of the plurality of documents of the first format into one or more individual pages. The method further comprises individually parsing the one or more individual pages of the plurality of documents. The parsing comprises identifying a predefined set of items of the one or more individual pages. Further processing comprises gathering the predefined set of items of each of the one or more individual pages of the plurality of documents into individual page files of a second format and performing the document processing service with the individual page files of the second format. The invention further concerns a corresponding computing system and a related computer program product.

Memory-efficient document processing

Computer-implemented methods and apparatus are provided for annotating digital images of line plots with ground truth labels. For each digital image, such a method includes supplying image data defining the image of a line plot to a machine-learning model trained to generate a set of control points defining a spline corresponding to the line plot. The method further comprises displaying the spline, and the set of control points, superimposed on the image in a graphical user interface and, in response to user manipulation via the graphical user interface of one or more control points, dynamically adjusting the displayed spline in accordance with manipulated control points whereby the displayed spline can be adjusted for conformity with the line plot. The set of control points for the adjusted spline is then stored as a ground truth label for the image.

Annotation of digital images for machine learning

A computer-implemented method for generating ground-truth for natural language querying may include providing a knowledge graph as data model, receiving a natural language query from a user and translating the natural language query into a formal data query. The method can also include visualizing the formal data query to the user and receiving a feedback response from the user. The feedback response can include a verified and/or edited formal data query. The method can also include storing the natural language query and the corresponding feedback response as ground-truth pair. Corresponding system and a related computer program product may be provided.

Translating a natural language query into a formal data query

A plurality of electronic documents comprising one or more document pages are received. First position markers, second position markers and page identifiers are inserted to the pages. The plurality of electronic documents are printed, thereby generating a printed corpus comprising a plurality of printed documents. The plurality of printed documents are scanned, thereby generating a scanned corpus comprising a plurality of scanned images. Scanning frame positions of the first and the second position markers are detected and the detected scanning frame positions and the page positions are used to define affine transformations between the plurality of scanned images and the corresponding document pages. The affine transformations are applied to the plurality of scanned images to align the plurality of scanned images with the corresponding document pages of the plurality of electronic documents.

Ground truth generation from scanned documents

A computer-implemented method for digitizing a document, wherein the document has assigned a classification scheme may be provided. A digital image and an identifier of the classification scheme may be received, the image representing a portion of the document. A segmentation of the image may be determined into one or more image segments; for each of the image segments, content information may be captured from the image segment and a category may be assigned to the image segment, the category being selected from the classification scheme. One or more digitization segments may be selected from the segmentation. A graph model of the document may be populated, wherein each of the digitization segments is represented by a segment node of the graph model.

Digital image-based document digitization using a graph model

A method of collecting training data of a document component may be provided. The documents have a structure and are coded in the typesetting language TeX. The method comprise receiving a TeX source file, compiling it into a PDF file and a related sync file, analyzing the PDF file, thereby determining a non-text-only document component. The method comprises also determining first coordinates of the non-text-only document component and a corresponding page number, determining a typesetting command relating to a non-text-only document component and determining second coordinates of a bounding box and a corresponding page number from the sync file, determining text elements in the non-text-only document component of the PDF file for which the first coordinates and the second coordinates overlap, and combining the determined text elements and linking them to a type of a non-text document component determined in the non-text-only document component in the TeX source file.

Growing community of inventors

Zurich, Switzerland

Michele Dolfi