International Business Machines Corporation

Vishwajeet Kumar

Jaydeep Sen

Saneem Ahmed Chemmengath

Samarth Bharadwaj

Mustafa Canim

Ioannis Katsis

Retrieval aware natural language question generation for open domain document retrieval is provided. In one aspect, a system for retrieval aware question generation includes: a question decontextualizer configured to decontextualize a question generated from a context of a target document by adding terms from the context into the question itself to create a decontextualized question, where the decontextualized question alone enables open domain document retrieval without a need for also providing the context. The system can also include a detect document identifier configured to find the terms in the context; and a retriever configured to retrieve documents from the corpus of documents using the decontextualized question. A method for retrieval aware question generation using the present system is also provided.

Retrieval aware question generation

Techniques for enhanced table and text question answering based on multi-instance, multi-answer training are presented. An answer extractor component can determine answer scores associated with candidate answer data items based on analysis of a set of data, comprising row data items of a table and passage data items associated with the table, and a context of a query of the set of data. The answer extractor component can be trained based on application of denoised single-instance and multiple-instance answer matching data associated with contexts to an answer extractor model to generate a trained answer extractor model of the answer extractor component. A query response component can determine a correct answer data item responsive to the query from the candidate answer data items based on the answer scores associated with the candidate answer data items, wherein the candidate answer data items can be reranked based on reweighted answer scores.

Multi-instance, multi-answer training for table and text question answering

A question answering bot that digests non-relational data tables is provided. A processor receives a question regarding a non-relational data table. A processor extracts at least one feature of the question using a natural language processing (NLP) model. A processor extracts at least one similar feature of the non-relational data table to the extracted at least one feature of the question. A processor determines at least one relevant cell in the non-relational data table based on the at least one feature of the question and the at least one similar feature of the non-relational data table. A processor provides an answer to the question, where the answer is based on the at least one relevant cell.

Natural language question answering using non-relational tables

Methods, systems, and computer program products for providing a contextual dialogue framework over dynamic tables are provided herein. A computer-implemented method includes maintaining a context space for a natural language conversation of a user, wherein the context space comprises a dynamic set of one or more tables used for processing at least one query of the natural language conversation; obtaining an additional table associated with an additional query of the natural language conversation; discovering one or more implicit links between the additional table and the dynamic set of tables; updating the context space with the one or more implicit links; and answering the additional query based at least in part on the updated context space.

Growing community of inventors

Bengaluru, India

Vishwajeet Kumar