International Business Machines Corporation

Md Faisal Mahbub Chowdhury

Alfio Massimiliano Gliozzo

Michael Robert Glass

Gaetano Rossiello

Nicolas Rodolfo Fauceglia

Sarthak Dash

Ruchi Mahindru

Robert G Farrell

Nandana Sampath Mihindukulasooriya

Yu Deng

Nidhi Rajshree

Anish Mathur

Biying Kong

Soumitra Sarkar

Sharon Mary Trewin

Adam P Lally

Abhijit Mishra

Dan Gutfreund

Nicholas Brady Garvan Monath

Md Arafat Sultan

Feifei Pan

Sagar Manohar

Chao Shang

Using a trained keyphrase generation model, a set of keyphrases corresponding to an input document is generated, a keyphrase in the set of keyphrases comprising a word summarizing a portion of a document. A relevance score measuring a similarity between the keyphrase and the document is calculated for a keyphrase in the set of keyphrases. The relevance score is adjusted according to a diversity balancing function.

Keyphrase generation

Determining an initial rank and a probability of relevance of each of a retrieved plurality of electronic documents relevant to a query. For each of a plurality of candidate facets, determine a revised rank for each of the retrieved plurality of electronic documents relevant to the query. Selecting, for each of the retrieved plurality of electronic documents relevant to the query, a minimum rank from among the initial rank and the revised rank for each of the plurality of candidate facets. Determine an expected discounted cumulative gain based on the probability of relevance and the minimum rank for each of the retrieved plurality of electronic documents relevant to the query. Select a set of optimistic facets based on maximizing the expected discounted cumulative gain.

Optimistic facet set selection for dynamic faceted search

One or more systems, devices, computer program products and/or computer-implemented methods of use provided herein relate to a process for generating the classification of files to allow for file system organization and/or query augmentation. A system can comprise a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory, wherein the computer executable components can comprise a generating component that generates a keyphrase based on a context derived from evaluation of an input file, wherein the generating component employs a public repository of files annotated with a plurality of keyphrases, including the keyphrase, to generate the keyphrase based on the context, and an execution component that classifies the input file based on the keyphrase. In one or more embodiments, the input file can comprise a query, and classification of the input file can comprise augmenting the query based on the keyphrase.

Keyphrase generation leveraging public repository categories

Embodiments of the present invention provide a computer system, a computer program product, and a method that comprises analyzing a performed query by identifying a plurality of indicative markers based on a pre-stored classification database associated with the performed query; generating a plurality of facets based on the analysis of the performed query; selecting at least two facets within the generated plurality of facets by determining a quantitative similarity value between each respective facet and the plurality of identified indicative markers associated with the performed query; dynamically ranking the selected facets by prioritizing the selected facets based on a calculated overall score associated with assigned weighted values for each selected facet in the generated plurality of facets using a supervised machine learning algorithm; and displaying the dynamically ranked facets within a user interface of a computing device associated with a user.

Dynamic facet ranking

With a computerized search engine, retrieve a plurality of electronic documents relevant to a query. Obtaining, via computerized term embedding, from the retrieved documents, a plurality of most similar terms with respect to the query. For each of the most similar terms, determine a pervasiveness score and a relevance score. Filter out, from the most similar terms, those of the terms that are pervasive, based on the pervasiveness score, those of the terms that are irrelevant, based on the relevance score, and those of the terms that are redundant. Output a top number of terms remaining in the most similar terms after the filtering, based on similarity to the query, as discriminative facets.

Unsupervised discriminative facet generation for dynamic faceted search

One embodiment of the present invention provides a method comprising receiving a text corpus, and generating a first list of triples based on the text corpus. Each triple of the first list comprises a first term representing a candidate hyponym, a second term representing a candidate hypernym, and a frequency value indicative of a number of times a hypernymy relation is observed between the candidate hyponym and the candidate hypernym in the text corpus. The method further comprises training a neural network for hypernym induction based on the first list. The trained neural network is a strict partial order network (SPON) model.

Hypernym detection using strict partial order networks

In an approach to unsupervised corpus expansion using domain-specific terms, one or more computer processors retrieve one or more domain-specific terms from a corpus of text. One or more computer processors search the World Wide Web for the one or more domain-specific terms to produce a plurality of web pages associated with each of the one or more domain-specific terms. One or more computer processors determine a confidence score for each of the plurality of web pages. One or more computer processors determine the confidence score of at least one of the plurality of web pages exceeds a pre-defined threshold. One or more computer processors add the at least one of the plurality of web pages to the corpus of text.

Unsupervised corpus expansion using domain-specific terms

An embodiment of the present invention generates natural language content from a set of keywords in accordance with a template. Keyword vectors representing a context for the keywords are generated. The keywords are associated with language tags, while the template includes a series of language tags indicating an arrangement for the generated natural language content. Template vectors are generated from the series of language tags of the template and represent a context for the template. Contributions from the contexts for the keywords and the template are determined based on a comparison of the series of language tags of the template with the associated language tags of the keywords. One or more words for each language tag of the template are generated to produce the natural language content based on combined contributions from the contexts for the keywords and the template.

Natural language text generation from a set of keywords using machine learning and templates

One embodiment of the invention provides a method for terminology ranking for use in natural language processing. The method comprises receiving a list of terms extracted from a corpus, where the list comprises a ranking of the terms based on frequencies of the terms across the corpus. The method further comprises accessing a domain ontology associated with the corpus, and re-ranking the list based on the domain ontology. The resulting re-ranked list comprises a different ranking of the terms based on relevance of the terms using knowledge from the domain ontology. The method further comprises generating clusters of terms via a trained model adapted to the corpus, and boosting a rank of at least one term of the re-ranked list based on the clusters to increase a relevance of the at least one term using knowledge from the trained model.

Discovering ranked domain relevant terms using knowledge

Training a machine learning model such as a neural network, which can automatically extract a hypernym from unstructured data, is disclosed. A preliminary candidate list of hyponym-hypernym pairs can be parsed from the corpus. A preliminary super-term&#x2014;sub-term glossary can be generated from the corpus, the preliminary super-term&#x2014;sub-term glossary containing one or more super-term&#x2014;sub-term pairs. A super-term&#x2014;sub-term pair can be filtered from the preliminary super-term&#x2014;sub-term glossary, responsive to detecting that the super-term&#x2014;sub-term pair is not a candidate for hyponym-hypernym pair, to generate a final super-term&#x2014;sub-term glossary. The preliminary candidate list of hyponym-hypernym pairs and the final super-term&#x2014;sub-term glossary can be combined to generate a final list of hyponym-hypernym pairs. An artificial neural network can be trained using the final list of hyponym-hypernym pairs as a training data set, the artificial neural network trained to identify a hypernym given new text data.

Unsupervised hypernym induction machine learning

An approach to induction of unknown terms into a term taxonomy graph may be provided. The approach may include analyzing a domain specific corpus to generate a term taxonomy graph using a term taxonomy graph generation model with a term knowledge base and determining which terms within the domain specific corpus are out of vocabulary (OOV) terms. The approach may also analyze the terms in the domain specific corpus with a semantic representation model to generate feature vectors of the OOV terms and terms known within the generated term taxonomy graph. The approach may determine if an OOV can be a hyponym of a term within the term taxonomy graph based on the feature vectors and insert the OOV term into the graph at the appropriate location.

Taxonomy generation to insert out of vocabulary terms and hypernym-hyponym pair induction

A method, system, and computer program product for automated evaluation of information retrieval systems are provided. The method accesses a natural language query from a set of natural language queries. The natural language query is associated with a query difficulty level. The method generates one or more natural language responses to the natural language natural language query. Each natural language response is associated with at least one facet of the plurality of facets. The method generates a set of feedback cues. A set of search results for the natural language query are returned. The set of search results include a highest ranked natural language response of the one or more natural language responses. The method generates an evaluation result for the HCIR system for the query difficulty level based on the one or more natural language responses, the set of search results, and the set of feedback cues.

Automated evaluation of information retrieval

A system, computer program product, and method are provided for employing a graph neural network (GNN) to construct a taxonomy. The GNN is subject to a training cycle and an inference cycle. The training cycle encodes cross-domain terms pairs from a set of noisy cross domain pairs extracted from a corpora, and outputs a preliminary taxonomy. The inference cycle identifies candidate term pairs and selectively subjects the candidate term pairs to selective filtering to produce a system predicted taxonomy from the preliminary taxonomy.

Taxonomy construction via graph-based cross-domain knowledge transfer

A query-focused faceted structure generation method, system, and computer program product for generating a query-focused faceted structure from a taxonomy for searching a document collection, including ingesting a document corpus, generating a vector space representation of a query and instances from a taxonomy of the document corpus, and producing a dynamic structure of a relevant facet categories and facet values using a two-vector space representation from the generated vector space representation.

Growing community of inventors

Woodside, NY, United States of America

Md Faisal Mahbub Chowdhury