International Business Machines Corporation

Nicholas Brady Garvan Monath

Robert G Farrell

Michael Robert Glass

Alfio Massimiliano Gliozzo

Md Faisal Mahbub Chowdhury

Gaetano Rossiello

Md Arafat Sultan

A computer-implemented method for performing cross-document coreference for a corpus of input documents includes determining mentions by parsing the input documents. Each mention includes a first vector for spelling data and a second vector for context data. A hierarchical tree data structure is created by generating several leaf nodes corresponding to respective mentions. Further, for each node, a similarity score is computed based on the first and second vectors of each node. The hierarchical tree is populated iteratively until a root node is created. Each iteration includes merging two nodes that have the highest similarity scores and creating an entity node instead at a hierarchical level that is above the two nodes being merged. Further, each iteration includes computing the similarity score for the entity node. The nodes with the similarity scores above a predetermined value are entities for which coreference has been performed in input documents.

Encoding entity representations for cross-document coreference

Training a machine learning model such as a neural network, which can automatically extract a hypernym from unstructured data, is disclosed. A preliminary candidate list of hyponym-hypernym pairs can be parsed from the corpus. A preliminary super-term&#x2014;sub-term glossary can be generated from the corpus, the preliminary super-term&#x2014;sub-term glossary containing one or more super-term&#x2014;sub-term pairs. A super-term&#x2014;sub-term pair can be filtered from the preliminary super-term&#x2014;sub-term glossary, responsive to detecting that the super-term&#x2014;sub-term pair is not a candidate for hyponym-hypernym pair, to generate a final super-term&#x2014;sub-term glossary. The preliminary candidate list of hyponym-hypernym pairs and the final super-term&#x2014;sub-term glossary can be combined to generate a final list of hyponym-hypernym pairs. An artificial neural network can be trained using the final list of hyponym-hypernym pairs as a training data set, the artificial neural network trained to identify a hypernym given new text data.

Unsupervised hypernym induction machine learning

Growing community of inventors

North Hampton, MA, United States of America

Nicholas Brady Garvan Monath