International Business Machines Corporation

Prithviraj Sen

Yunyao Li

Shivakumar Vaithyanathan

Rajasekar Krishnamurthy

Berthold Reinwald

Marina Danilevsky Hailpern

Lucian Popa

Laura Chiticariu

Douglas Ronald Burdick

Min Li

Hao Wang

Yuanyuan Tian

Shirish Tatikonda

Alexandre Valentinovich Evfimievski

Matthias Boehm

Kun Qian

Sang Don Han

Dakshi Agrawal

Mu Qiao

Jeffrey Thomas Kreulen

Eser Kandogan

Huaiyu Zhu

Shajith I Mohamed

Frederick Ralph Reiss

Yuya Jeremy Ong

Saeed Ghanbari

Siddhartha Brahma

Sairam Gurajada

Sheshera Mysore

Shivam Srivastava

Yiwei Yang

Poornima Chozhiyath Raman

Xilun Chen

Wei Cheng

Zihui Li

A method, system, and computer program product for learning entity resolution rules for determining whether entities are matching. The method may include receiving historical pairs of entities. The method may also include determining a set of rules for determining whether a pair of entities are matching, where the set of rules comprises a plurality of conditions. The method may also include developing, using a deep neural network, an entity resolution model based on the historical pairs of entities. The method may also include receiving a new pair of entities. The method may also include applying the entity resolution model to the new pair of entities. The method may also include determining whether one or more rules from the set of rules are satisfied for the new pair of entities. The method may also include categorizing the new pair of entities as matching or not matching.

Deep learning of entity resolution rules

A system, computer program product, and method are provided for jointly learning dictionary based rules and dictionary candidates. Natural language text is received and parsed into subsets, with the subset being subjected to natural language processing to identify one or more verbs within the subset. The identified verbs are evaluated with respect to a dictionary and one or more rules. The evaluation is directed at each predicate in the rules with respect to the identified verbs. A neural network is leveraged to jointly induce modification of the rules and one or more dictionaries responsive to the evaluation.

Learning rules and dictionaries with neuro-symbolic artificial intelligence

In an approach for forecasting in multivariate irregularly sampled time series, a processor receives time series data having one or more missing values. A processor determines, from the time series data, non-missing values present in the time series data. A processor determines, from the time series data, zero or more mask values for the time series data. A processor determines time interval values. A processor inputs the one or more missing values, the non-missing values, the zero or more mask values, and the time interval values into a recurrent neural network. A processor determines a predicted value for the one or more missing values.

Forecasting in multivariate irregularly sampled time series with missing values

A computer-implemented method is provided for predicting future data values or target labels of multivariate time series data. The method includes receiving the multivariate time series data having present values, systematic missing values, and random missing values. The method further includes masking the present values, the systematic missing values, and the random missing values using triplet encodings. The method also includes determining time intervals between current missing values, from among the systematic missing values and the random missing values, and immediately preceding ones of the present values. The method additionally includes training, by a computing device, at least one recurrent neural network with the triplet encodings, the time intervals, and multivariate time series data to perform a feedforward pass on the recurrent neural network predicting the future data values or the target labels. A forward imputation is utilized by the computing device to impute any of the current missing values.

Predicting multivariate time series with systematic and random missing values

A computer-implemented method according to one embodiment includes receiving a plurality of linguistic expressions (LEs); changing one or more conditions of the plurality of linguistic expressions to create an updated plurality of linguistic expressions, utilizing a visual exploration framework (VEF) that visually presents to a user each of the plurality of linguistic expressions; and including the updated plurality of linguistic expressions in a model used to classify input sentences. According to another embodiment, a computer-implemented method includes receiving (i) a set of linguistic expressions (LEs) and (ii) a set of labeled data as input, where the LEs are logical combinations of predicates learned from the labeled data, and each data point in the labeled data comprises a piece of text and ground-truth labels; presenting the LEs in a visual exploration framework; and allowing a user to sort, filter, subset, and select LEs based on different criteria, utilizing the framework.

Adjusting explainable rules using an exploration framework

A computer-implemented method according to one embodiment includes receiving, at a deep neural network (DNN), a plurality of sentences each having an associated label; training the DNN, utilizing the plurality of sentences and associated labels; and producing a linguistic expression (LE) utilizing the trained DNN.

Producing explainable rules via deep learning

Providing a predictive model for a target language by determining an instance weight for a labeled source language textual unit according to a set of unlabeled target language textual units, scaling, by the one or more computer processors, an error between a predicted label for the source language textual unit and a ground-truth label for the source language textual unit according to the instance weight, updating, by the one or more computer processors, network parameters of a predictive neural network model for the target language according to the error, and providing, by the one or more computer processors, the predictive neural network model for the target language to a user.

Zero-shot cross-lingual transfer learning

One embodiment provides for a method for evaluation of an artificial intelligence (AI) service, the method includes partitioning, by a processor, data into in-domain data and out-of-domain data. The processor defines held-out data from both of the in-domain data and the out-of-domain data for evaluation by each of domain and sub-domain based on building a taxonomy of both domains and sub-domains for the AI service. The held-out data is excluded from training data used for training the AI service. The processor further determines distribution underlying performance metrics for the held-out data using bootstrap validation processing. The processor also determines performance guarantees for multiple settings conditioned on multiple characteristics of an application scenario for the held-out data of the taxonomy based on the underlying performance metrics.

Workflow for evaluating quality of artificial intelligence (AI) services using held-out data

Methods, systems, and computer program products for extracting structure and semantics from tabular data are provided herein. A computer-implemented method includes processing tabular data comprising data cells and header cells, wherein the processing includes: identifying one or more regions within the tabular data, wherein each of the regions comprises one or more of the data cells; matching some of the regions to one or more of the header cells, wherein the matched header cells are semantically related to the data cells inside the matched region; and generating, based on the matching, an output describing semantic relationships between the data cells and the header cells. The method also includes creating, for each data cell, a tuple comprising semantic information contained within one or more of the header cells that pertains to the data cell.

Extracting structure and semantics from tabular data

In an approach, a processor trains a statistical classifier and a set of micro classifiers. A processor receives an input to be classified by the statistical classifier. A processor receives a label assigned to the input by the statistical classifier and respective labels assigned by each micro classifier of the set of micro classifiers. A processor determines that the label assigned by the statistical classifier is the same as at least one label assigned by at least one micro classifier of the set of micro classifiers. A processor generates a natural language explanation for assigning the label using the at least one micro classifier and the label. A processor outputs the label and the natural language explanation to a user of a computing device. A processor receives user feedback from the user in the form of an acceptance or a rejection of the natural language explanation.

Natural language explanation for classifier predictions

Methods, systems, and computer program products for learning models for entity resolution using active learning are provided herein. A computer-implemented method includes determining a set of data items related to a task associated with structured knowledge base creation, and outputting the set of data items to a user for labeling. Such a method also includes generating, based on a user-labeled version of the set of data items, a candidate model for executing the task, and one or more generalized versions of the candidate model. Additionally, such a method can also include generating a final model based on one or more iterations of analysis of the candidate model and analysis of the one or more generalized versions of the candidate model, and performing the task by executing the final model on one or more datasets.

Learning models for entity resolution using active learning

One embodiment provides for a method for evaluation of an artificial intelligence (AI) service, the method includes partitioning, by a processor, data into in-domain data and out-of-domain data. The processor defines held-out data from the in-domain data and the out-of-domain data for evaluation by domain and sub-domain based on building a taxonomy of domains and sub-domains for the AI service. The processor further determines distribution underlying performance metrics for the held-out data using statistical processing. The processor also determines performance guarantees for multiple settings conditioned on multiple characteristics of an application scenario for the held-out data of the taxonomy based on the underlying performance metrics. The processor further provides confidence intervals based on the performance guarantees.

Methods, systems, and computer program products for table recognition in PDF documents are provided herein. A computer-implemented method includes discretizing one or more contiguous areas of a PDF document; identifying one or more white-space separator lines within the one or more discretized contiguous areas of the PDF document; detecting one or more candidate table regions within the one or more discretized contiguous areas of the PDF document by clustering the one or more white-space separator lines into one or more grids; and outputting at least one of the candidate table regions as a finalized table in accordance with scores assigned to each of the one or more candidate table regions based on (i) border information and (ii) cell structure information.

Table recognition in portable document format documents

One embodiment provides for a method that includes performing, by a processor, active learning of large scale entity resolution using a distributed compute memoing cache to eliminate redundant computation. Link feature vector tables are determined for intermediate results of the active learning of large scale entity resolution. The link feature vector tables are managed by a two-level cache hierarchy.

Two level compute memoing for large scale entity resolution

One embodiment provides a method for developing a text analytics program for extracting at least one target concept including: utilizing at least one processor to execute computer code that performs the steps of: initiating a development tool that accepts user input to develop rules for extraction of features of the at least one target concept within a dataset comprising textual information; developing, using the rules for feature extraction, an evaluation dataset comprising at least one document annotated with the at least one target concept to be extracted by the text analytics program; creating, using the rules for feature extraction, a rule-based annotator to extract the at least one target concept; training, using the evaluation dataset, a machine-learning annotator to extract the at least one target concept within the dataset; combining the rule-based annotator and the machine learning annotator to form a combined annotator; evaluating, using the evaluation dataset, extraction performance of the combined annotator against a predetermined threshold; and publishing, when the extraction performance of the combined annotator exceeds the predetermined threshold, the combined annotator for use in an application that extracts the at least one target concept from a plurality of datasets.

Growing community of inventors

San Jose, CA, United States of America

Prithviraj Sen