International Business Machines Corporation

Paul Lewis Felt

Brendan Bull

Scott Robert Carrier

Dwi Sianto Mansjur

Andrew C Hicks

David Contreras

Mario J Lorenzo

Provided are a computer program product, system, and method for using domain specific vocabularies to spellcheck input strings. A determination is made of a determining a domain specific vocabulary comprising domain specific tokens for a user. The input strings in the input text from the user are processed to determine whether the input strings match the domain specific tokens in the domain specific vocabulary. Indication is made that input strings that are determined to match the domain specific tokens in the domain specific vocabulary are not to be spellchecked by a global vocabulary available to users across domains.

Using domain specific vocabularies to spellcheck input strings

Techniques for concept disambiguation for natural language processing are described herein. An aspect includes receiving a message from a user. Another aspect includes identifying an ambiguous concept in the message. Another aspect includes determining a plurality of concept candidates corresponding to the ambiguous concept. Another aspect includes determining, for each of the plurality of concept candidates, a respective similarity score based on user-specific concept metrics corresponding to the user. Another aspect includes ranking the plurality of concept candidates based on the respective similarity scores. Another aspect includes determining that the ambiguous concept corresponds to a top-ranked concept candidate of the ranked plurality of concept candidates.

Concept disambiguation for natural language processing

The exemplary embodiments disclose a system and method, a computer program product, and a computer system for resolving the intents of user expression. The exemplary embodiments may include receiving a user expression, receiving a first resolver having an input class and an output class based on the user expression, determining whether the first resolver can be resolved based on the user expression, and based on determining that the first resolver can be resolved based on the user expression, resolving the first resolver.

Resolving user expression having dependent intents

Techniques for targeted partial re-enrichment include determining that at least one natural language processing (NLP) request is associated with at least one surface form, the NLP request being for a corpus, a database comprising preexisting annotations associated with the corpus. An index query related to the at least one surface form is performed to generate index query results, the index query results including identification of portions of the corpus affected by the NLP request. A scope of the NLP request related to the database is determined based on the index query results, the scope including identification of impacted candidate annotations of the preexisting annotations affected by the NLP request. An NLP service is performed on the corpus according to the scope and the portions, thereby resulting in updates. The updates are committed to the database associated with the corpus.

Targeted partial re-enrichment of a corpus based on NLP model enhancements

The exemplary embodiments disclose a system and method, a computer program product, and a computer system for determining the intents of user expression. The exemplary embodiments may include receiving a user expression, extracting one or more entities from the user expression, gathering one or more resolvers associated with the one or more entities, identifying a first resolver of the one or more resolvers based on the user expression and the one or more training expressions, and resolving the first resolver to generate a first output.

Leveraging intent resolvers to determine multiple intents

Embodiments include cross-document propagation of entity metadata. Aspects include identifying a set of documents from a plurality of documents, the set of documents being related to one another and identifying a concept in a first document of the set of documents and creating an annotation corresponding to the concept. Aspects also include evaluating the annotation from the first document against all of the documents in the set of documents and identifying a concept match between the annotation and a mention discovered in a second document in the set of documents. Aspects further include creating a metadata linkage between the concept in the first document to the mention in the second document.

Cross-document propagation of entity metadata

Examples described herein provide a computer-implemented method that includes receiving, by a processing device, the span of text, the span of text comprising a plurality of elements including at least an entity element and a temporal element. The method further includes organizing, by the processing device, the span of text as a natural language processing (NLP) parse tree. The method further includes traversing, by the processing device, the NLP parse tree by concatenating individual nodes of the span of text to generate the relation type between the entity element and the temporal element. The method further includes associating, by the processing device, the entity element, the relation type, and the temporal element together.

Inferring relation types between temporal elements and entity elements

A computer-implemented method, system and computer program product for automatically bootstrapping a domain-specific vocabulary from at least one source document using one or more computers, by: (a) encoding one or more passages in the source document to identify one or more relevant words therein, wherein the encoding assigns an importance to the relevant words using an attention mechanism (AM) on top of a recurrent neural network (RNN); (b) expanding the relevant words using word embedding distance, ontology information, or multi-part analogies; and (c) mapping the expanded words to concepts for inclusion into the domain-specific vocabulary, wherein concept disambiguation is performed to ensure that incorrect concepts are not included into the domain-specific vocabulary.

Automatically bootstrapping a domain-specific vocabulary

Aspects of the invention include a method for iteratively expanding concepts. The method includes building a set of expressions extracted from an ontology to form a cache, the set of expressions based at least in part on respective target concepts. Receiving a document and performing a first traversal of the document to identify first surface forms related to the respective target concepts. Performing a second traversal of the document to identify second surface forms that modify the first surface forms. Annotating the document by comparing the modifying surface forms to target concepts and the set of expressions in the cache.

Iteratively expanding concepts

Methods, systems, and computer program products for expanded concept matching are provided. Aspects include receiving an ontology, determining a set of target concepts, building a cache from the ontology, the cache comprising a set of expressions extracted from the ontology for each target concept in the set of target concepts, receiving a document, determining a first segment of text in the document based on the set of target concepts, and annotating the first segment of text by comparing the set of target concepts to the set of expressions in the cache.

Expanded concept matching

Examples described herein provide a computer-implemented method that includes receiving a ground truth associated with a domain cartridge, the domain cartridge comprising a plurality of hierarchical layers. The method further includes analyzing annotation blocks in relation to data present in the ground truth to detect any errors in a set of natural language processing annotators. The analyzing includes computing a recall score, a precision score, and an F1 score for each annotation block in a lowest level layer of the plurality of hierarchical layers. The analyzing further includes determining whether an error is detected at the lowest level layer of the plurality of hierarchical layers based at least in part on the recall score, the precision score, and the F1 score. The analyzing further includes terminating the analyzing responsive to determining that the error is detected at the lowest level layer of the plurality of hierarchical layers.

Ascribing ground truth performance to annotation blocks

Aspects of the invention include converting text from a first image file into a first machine-encodable text, wherein the image file includes a first section of text that is offset from a second section of text. Analyzing the first image file to detect a position of the first section of text. Embedding a first section of the first machine encodable-text with metadata describing the position of the first section of text. Reformatting the first section of the first machine encodable-text to conform to the position of the first section of text.

Enhanced section detection using a combination of object detection with heuristics

A computer system generates a vector space model based on an ontology of concepts. One or more training examples are extracted for one or more concepts of a hierarchical ontology, wherein the one or more training examples for the one or more concepts are based on neighboring concepts in the hierarchical ontology. A plurality of vectors, each including one or more features, are initialized, wherein each vector corresponds to a concept of the one or more concepts. A vector space model is generated by iteratively modifying one or more vectors of the plurality of vectors to optimize a loss function. Natural language processing is performed using the vector space model. Embodiments of the present invention further include a method and program product for generating a vector space model in substantially the same manner described above.

Natural language processing using an ontology-based concept embedding model

Aspects of the invention include systems and methods for the propagation of annotation metadata to overlapping annotations of a synonymous type. A non-limiting example computer-implemented method includes performing a comparison of a set of annotations to detect a subset of annotations that are candidates of being synonymous based on a first analysis. Whether a first annotation of the subset of annotations is synonymous with a second annotation of the subset of annotations is determined based on a second analysis. Distinct annotation metadata of the first annotation are cross-propogated with annotation metadata of the second annotation based on the second analysis.

Growing community of inventors

Springville, UT, United States of America

Paul Lewis Felt