International Business Machines Corporation

Rensselaer Polytechnic Institute

Lucian Popa

Mauricio Antonio Hernandez-Sherrington

Ching-Tien Ho

Ronald Fagin

Phokion Kolaitis

Oktie Hassanzadeh

Rajasekar Krishnamurthy

Prithviraj Sen

Wang-Chiew Tan

Kun Qian

Mauricio A Hernandez

Min Li

Georgia Koutrika

Craig Salter

Armageddon Rhabdizo Brown

Suresh Thalamati

Sairam Gurajada

Ryan Wisnesky

Cong Yu

Nitin Gaur

Yunyao Li

Shivakumar Vaithyanathan

Yada Zhu

Mary Ann Roth

Sijia Liu

Kumar Bhaskaran

Hima Prasad Karanam

Douglas Ronald Burdick

Lingling Yan

Laura Myers Haas

Renee J Miller

Bogdan Alexe

Koushik Kar

Wook-Shin Han

Mary Anne Roth

Jungo Kasai

Aparna Gupta

Sai Radhakrishna Manikant Sarma Palepu

Li Qian

Sheshera Mysore

Haifeng Jiang

Laura Chiticarlu

Felix Gerhard Naumann

A graph neural network can be built and trained to predict a risk of an entity. A multi-relational graph network can include a first graph network and a second graph network. The first graph network can include a first set of nodes and a first set of edges connecting some of the nodes in the first set. The second graph network can include a second set of nodes and a second set of edges connecting some of the nodes in the second set. The first set of nodes and the second set of nodes can represent entities, the first set of edges can represent a first relationship between the entities and the second set of edges can represent a second relationship between the entities. A graph convolutional network (GCN) can be structured to incorporate the multi-relational graph network, and trained to predict a risk associated with a given entity.

Multi-relational graph convolutional network (GCN) in risk prediction

A method, system, and computer program product for learning entity resolution rules for determining whether entities are matching. The method may include receiving historical pairs of entities. The method may also include determining a set of rules for determining whether a pair of entities are matching, where the set of rules comprises a plurality of conditions. The method may also include developing, using a deep neural network, an entity resolution model based on the historical pairs of entities. The method may also include receiving a new pair of entities. The method may also include applying the entity resolution model to the new pair of entities. The method may also include determining whether one or more rules from the set of rules are satisfied for the new pair of entities. The method may also include categorizing the new pair of entities as matching or not matching.

Deep learning of entity resolution rules

Methods, systems, and computer program products for low-resource entity resolution with transfer learning are provided herein. A computer-implemented method includes processing input data via a first entity resolution model, wherein the input data comprise labeled input data and unlabeled input data; identifying one or more portions of the unlabeled input data to be used in training a neural network entity resolution model, wherein said identifying comprises applying one or more active learning algorithms to the first entity resolution model; training, using (i) the one or more portions of the unlabeled input data and (ii) one or more deep learning techniques, the neural network entity resolution model; and performing one or more entity resolution tasks by applying the trained neural network entity resolution model to one or more datasets.

Low-resource entity resolution with transfer learning

Data records are linked across a plurality of datasets. Each dataset contains at least one data record, and each data record is associated with an entity and includes one or more attributes of that entity and a value for each attribute. Values associated with attributes are compared across datasets, and matching attributes having values that satisfy a predetermined similarity threshold are identified. In addition, linkage points between pairs of datasets are identified. Each linkage point links one or more pairs of data records. Each data record in the pair of data records is contained in one of a given pair of datasets, and each pair of data records is associated with a common entity having matching attributes in the given pair of datasets. Data records associated with the common entities are linked across datasets using the identified linkage points.

Discovery of linkage points between data sources

Methods, systems, and computer program products for learning models for entity resolution using active learning are provided herein. A computer-implemented method includes determining a set of data items related to a task associated with structured knowledge base creation, and outputting the set of data items to a user for labeling. Such a method also includes generating, based on a user-labeled version of the set of data items, a candidate model for executing the task, and one or more generalized versions of the candidate model. Additionally, such a method can also include generating a final model based on one or more iterations of analysis of the candidate model and analysis of the one or more generalized versions of the candidate model, and performing the task by executing the final model on one or more datasets.

Learning models for entity resolution using active learning

One embodiment provides for a method that includes performing, by a processor, active learning of large scale entity resolution using a distributed compute memoing cache to eliminate redundant computation. Link feature vector tables are determined for intermediate results of the active learning of large scale entity resolution. The link feature vector tables are managed by a two-level cache hierarchy.

Two level compute memoing for large scale entity resolution

Methods and systems for discovery of linkage points between data sources

Described herein are methods, systems and computer program products for entity resolution. Entity resolution, also known as entity matching or record linkage, seeks to identify equivalent data objects between or among datasets. An example method includes creating a deterministic model by defining an entity to be resolved, selecting two datasets for comparison, defining matching predicates for attributes of the datasets to select a set of candidate matches, and defining a precedence rule for the candidate matches to select a subset of the candidate matches. The method includes running the deterministic model on the two datasets. Running the deterministic model includes applying the matching predicates and the precedence rule to data in the datasets that correspond to the attributes. The method also includes applying a cardinality rule to results of the running, and outputting the matching candidates for which the cardinality rule is satisfied.

Entity resolution between datasets

Embodiments of the present invention relate to a new method of entity integration using high-level scripting languages. In one embodiment, a method of and computer product for entity integration is provided. An entity declaration is read from a machine readable medium. The entity declaration describes an entity including at least one nested entity. An index declaration is read from a machine readable medium. The index declaration describes an index of nested entities. An entity population rule is read from a machine readable medium. The entity population rule describes a mapping from an input schema to an output schema. The output schema conforms to the entity declaration. A plurality of input records is read from a first data store. The input records conform to the input schema. The entity population rule applies to the plurality of records to create a plurality of output records complying with the output schema. An index of nested entities is populated. The index complies with the index declaration.

Entity integration using high-level scripting languages

Embodiments of the present invention relate to a declarative framework for efficient incremental information integration. In one embodiment, a method of and computer program product for information integration is provided. An integration rule is received. A first data set is accessed. A first representation of the first data is generated set based on the plurality of integration rules. The first representation is flat and includes a plurality of records. At least one index is generated. The index encodes at least one hierarchical relationship among the plurality of records. A second representation is generated of the first representation based on the at least one index. The second representation comprising nested data.

Incremental information integration using a declarative framework

A computer-implemented method for use with an extensible markup language (XML) document includes inputting a high-level mapping specification for a schema mapping; and generating a target XML document based on the mapping. The method may perform schema mapping-based XML transformation as a three-phase process comprising tuple extraction, XML-fragment generation, and data merging. The tuple extraction phase may be adapted to handle streamed XML data (as well as stored/indexed XML data). The data merging phase may use a hybrid method that can dynamically switch between main memory-based and disk-based algorithms based on the size of the XML data to be merged.

Scalable algorithms for mapping-based XML transformation

Methods, systems and computer program products for interactive generation of integrated schemas. Exemplary embodiments include a method for schema integration, the method including recasting a first source schema into a first graph of concepts with HasA relationships, recasting a second source schema into a second graph of concepts with HasA relationships, identifying matching concepts in the first graph and the second graph based on correspondences between attributes of the concepts of the first and second graphs, producing an integrated schema, based on a fixed specification of matching concepts to merge, and generating a mapping from the first source schema to the integrated schema and from the second source schema to the integrated schema.

Interactive generation of integrated schemas

A method, system and program product for specifying, in a schema mapping framework, a mapping between a source schema and a target schema. The source and target schemas are schemas included in respective groups of registered, heterogeneous schemas. The source and target schemas may be of different types. Serialized versions of the source and target schemas include source objects and target objects, respectively. A mapping model is serialized into mapping objects that include logical references representing the source objects and logical references representing the target objects. The logical references are resolved to the source objects and target objects, thereby storing pointers to the source objects and to the target objects. After resolving the logical references, the mapping model includes the logical references and the pointers to the source and target objects.

Schema mapping specification framework

Growing community of inventors

San Jose, CA, United States of America

Lucian Popa