International Business Machines Corporation

Esme Manandise

Branimir Konstantinov Boguraev

Benjamin P Segal

A computer program product for domain-specific data generation. The computer program product including a computer-readable storage medium having program instructions embodied therewith. The program instructions executable by a processor to cause the processor to perform operations including performing an analysis on a domain-specific corpus to identify a base term and a modifier term. The modifier term modifies the base term in at least a portion of the domain-specific corpus. The operations also include accessing a first entry in lexicon data. The first entry including core data corresponding to domain-independent lexical information for the base term. The operations further include adding, based on the analysis, non-core data to the first entry, the non-core data corresponding to domain-specific lexical information for the base term, wherein the non-core data identifies the modifier term as a domain-specific modifier of the base term.

Domain-specific lexical analysis

A method includes performing, at a device, an analysis of a domain-specific corpus to identify a base term and a modifier term. The modifier term modifies the base term in at least a portion of the domain-specific corpus. The method also includes accessing, by the device, a first entry in lexicon data. The first entry includes core data corresponding to domain-independent lexical information for the base term. The method further includes adding, based on the analysis, non-core data to the first entry. The non-core data corresponds to domain-specific lexical information for the base term. The non-core data identifies the modifier term as a domain-specific modifier of the base term.

A method includes obtaining an input text, identifying a first term in the input text, and accessing lexicon data to identify a first entry corresponding to the first term. The first entry includes core data corresponding to domain-independent lexical information for the first term, and non-core data corresponding to domain-specific lexical information for the first term. The method also includes determining that the non-core data of the first entry identifies a second term in the input text as a modifier of the first term. The method further includes generating a partially parsed and bracketed version of the input text. The partially parsed and bracketed version indicates that the second term modifies the first term in the input text. The method also includes generating a parsed version of the input text based on the partially parsed and bracketed version of the input text.

Domain-specific lexically-driven pre-parser

According to an aspect, a candidate token sequence including one or more word tokens is extracted from an unstructured domain glossary that includes entries associated with a domain. A look-up operation is performed to retrieve language data for each word token in the candidate token sequence and annotates each word token in the candidate token sequence found by the look-up operation with corresponding retrieved language data to form an annotated sequence. A pattern match of the annotated sequence is performed relative to a repository of patterns and identifies a best matching pattern from the repository of patterns to the annotated sequence based on matching criteria. The annotated sequence is refined with lexical information associated with the best matching pattern as a refined annotated sequence. The candidate token sequence and the refined annotated sequence are output to a domain-specific computational lexicon file.

Domain-specific computational lexicon formation

According to an aspect, a candidate lexical kernel unit that includes a word token sequence having two or more words is received. Domain terms that contain the two or more words are retrieved from a terminology resource file of domain terms associated with a domain. The candidate lexical kernel unit and the retrieved domain terms are analyzed to determine whether the candidate lexical kernel unit satisfies specified criteria for use as a building block by a natural-language processing (NLP) tool for building larger lexical units in the domain. Each of the larger lexical units includes a greater number of words than the candidate lexical kernel unit. The candidate lexical kernel unit is identified as a lexical kernel unit based on determining that the candidate lexical kernel unit satisfies the specified criteria. The lexical kernel unit is output to a domain-specific lexical kernel unit file for input to the NLP tool.

Growing community of inventors

Tallahassee, FL, United States of America

Esme Manandise