International Business Machines Corporation

Jaydeep Sen

Karthik Sankaranarayanan

Diptikalyan Saha

Ashish Mittal

Saneem Ahmed Chemmengath

Manasa Jammi

Vishwajeet Kumar

Tarun Tater

Vatche Isahagian

Samarth Bharadwaj

Vinod Muthusamy

Mustafa Canim

Ya Bin Dang

Vinayaka D Pandit

Sumit Bhatia

Sreyash Kenkre

Parag Jain

Yara Rizk

Xue Han

Ayushi Dalmia

Ioannis Katsis

Yazan Obeidi

Bu Yu Gao

Retrieval aware natural language question generation for open domain document retrieval is provided. In one aspect, a system for retrieval aware question generation includes: a question decontextualizer configured to decontextualize a question generated from a context of a target document by adding terms from the context into the question itself to create a decontextualized question, where the decontextualized question alone enables open domain document retrieval without a need for also providing the context. The system can also include a detect document identifier configured to find the terms in the context; and a retriever configured to retrieve documents from the corpus of documents using the decontextualized question. A method for retrieval aware question generation using the present system is also provided.

Retrieval aware question generation

A query can be received from a user. The query can be sent to a plurality of automated agents to process the query. Results and associated confidence scores can be received from the plurality of automated agents. At least some of the results and associated confidence scores can be probed, based at least on a reason given for a result having the highest associated confidence score among the received results and associated confidence scores, to select an automated agent from the plurality of automated agents for answering the query. Information can be stored, where the information can include at least the results and associated confidence scores and a selected automated agent for answering the query, where at least one of the plurality of automated agents learns from the stored information to update its confidence score in answering the query.

Adaptive answer confidence scoring by agents in multi-agent system

One or more systems, devices, computer program products and/or computer-implemented methods of use provided herein relate to a process to facilitate multi-lingual query interpretation. A system can comprise a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory, wherein the computer executable components can comprise an annotation component that generates one or more language invariant signals, an interpretation component that generates a complete query intent using the one or more language invariant signals, and a translation component that processes the complete query intent to an executable backend query to facilitate multi-lingual query interpretation. In one or more embodiments, the translation component can be operatively connected with the interpretation component to generate a zero-shot transfer of the one or more language invariant signals.

Multi-lingual natural language query

A natural language processing approach for generating domain specific reasoning-based meaning representations. The approach may include receiving a user query via structured or unstructured data. The approach may also include generating a structured query from the user query using domain specific ontology and universal facts. Further, the approach many include the structured query may be analyzed to determine if the structured query has been assigned consistent concepts, properties and actions. Additionally, the approach may involve correcting the structured query if it is determined the structured query is inconsistent with the domain ontology and the universal facts.

Reasoning based natural language interpretation

Techniques for enhanced table and text question answering based on multi-instance, multi-answer training are presented. An answer extractor component can determine answer scores associated with candidate answer data items based on analysis of a set of data, comprising row data items of a table and passage data items associated with the table, and a context of a query of the set of data. The answer extractor component can be trained based on application of denoised single-instance and multiple-instance answer matching data associated with contexts to an answer extractor model to generate a trained answer extractor model of the answer extractor component. A query response component can determine a correct answer data item responsive to the query from the candidate answer data items based on the answer scores associated with the candidate answer data items, wherein the candidate answer data items can be reranked based on reweighted answer scores.

Multi-instance, multi-answer training for table and text question answering

A question answering bot that digests non-relational data tables is provided. A processor receives a question regarding a non-relational data table. A processor extracts at least one feature of the question using a natural language processing (NLP) model. A processor extracts at least one similar feature of the non-relational data table to the extracted at least one feature of the question. A processor determines at least one relevant cell in the non-relational data table based on the at least one feature of the question and the at least one similar feature of the non-relational data table. A processor provides an answer to the question, where the answer is based on the at least one relevant cell.

Natural language question answering using non-relational tables

Mechanisms are provided for processing a sequential database natural language query. A process model is preprocessed to generate mapping data structure(s). The mapping data structure(s) map elements of the sequential process to other elements of the sequential process to thereby identify sequential and dependent characteristics of the sequential process. A sequential database natural language (SDNL) query interpretation engine is configured with the mapping data structure(s) and natural language processing is performed on a query to generate extracted features. The configured SDNL query interpretation engine operates on the extracted features and the mapping data structure(s) to generate intent information for the query. Executable database queries are generated based on the intent and executed on a database to return a response to the query.

Natural language querying of sequential process automation data

Methods, systems, and computer program products for providing a contextual dialogue framework over dynamic tables are provided herein. A computer-implemented method includes maintaining a context space for a natural language conversation of a user, wherein the context space comprises a dynamic set of one or more tables used for processing at least one query of the natural language conversation; obtaining an additional table associated with an additional query of the natural language conversation; discovering one or more implicit links between the additional table and the dynamic set of tables; updating the context space with the one or more implicit links; and answering the additional query based at least in part on the updated context space.

Contextual dialogue framework over dynamic tables

Methods, systems, and computer program products for active learning for natural language question answering are provided herein. A computer-implemented method includes generating a semantic signature of a natural language query; generating a SQL signature for a SQL query corresponding to the natural language query; determining whether a set of mappings includes a semantic signature matching the generated semantic signature, wherein each mapping in the set is between (i) a given semantic signature and (ii) a SQL signature representing a class of SQL queries corresponding to the given semantic signature; and in response to determining that the set of mappings does not include the generated semantic signature, adding a mapping between the generated semantic signature and the generated SQL signature to the set of mappings.

Active learning for natural language question answering

An embodiment for identifying and processing poly-process natural language queries may include receiving a natural language query. The embodiment may also automatically identify a bridge entity in the received natural language query. The embodiment may also automatically determine whether the received natural language query is a poly-process query. The embodiment may further include, in response to identifying that the received natural language query is the poly-process query, automatically generating sub-queries for each process in the poly-process query and generate results for each sub-query. The embodiment may also automatically combining the results of each sub-query using the bridge entity to output a combined result. The embodiment may further include automatically generating a modified sub-query for post-processing of the combined result. The embodiment may also automatically process the modified sub-query to generate a final query result for the received natural language query.

Identifying and processing poly-process natural language queries

Methods, systems, and computer program products for processing natural language analytics queries are provided herein. A computer-implemented method includes obtaining a natural language query comprising an analytics function; applying domain reasoning using a predefined grammar for a plurality of different predefined categories of analytics functions to assign the analytics function of the natural language query into a given analytics function category; identifying predefined arguments and a predefined sequence of actions associated with the given analytics function category; instantiating the analytics function using the predefined arguments and the predefined sequence of actions; interpreting the instantiated analytics function in the context of a domain ontology to generate a target executable query to implement the instantiated analytics function; and executing the predefined sequence of actions for the given analytics function class on a result of the target executable query to obtain an answer to the natural language query.

Natural language analytics queries

One embodiment provides a method, including: receiving a query from a user; identifying that a desired definition of the at least one term is unknown, by determining that the at least one term does not map to a term having a known definition; receiving the definition of the at least one term from the user; adding the definition to a domain grammar comprising (i) domain-specific terminology and (ii) definitions corresponding to the terms within the domain grammar, wherein the adding comprises (a) extracting expressions from the requested definition and (b) adding, for the at least one term, the expressions into a structured format within the domain grammar; combining (iii) the requested definition and (iv) terms from the parsed query having previously known definitions into a complete query; and providing a response to the query by executing the complete query on a knowledge store.

Domain query execution using user-provided definition

Embodiments are disclosed for correcting a natural language interface database (NLIDB) system. The techniques include receiving feedback indicating that an answer provided in response to a question for an NLIDB system is inaccurate. The techniques further include finding an ontology element for a datastore of the NLIDB system that matches to the feedback. The techniques also include selecting candidate annotations for the NLIDB system based on the ontology element and a data type of the ontology element. Additionally, the techniques include generating a question-answer (QA) pair for each of the candidate annotations. Further, the techniques include adding one of the candidate annotations to annotations for a natural language query (NLQ) engine of the NLIDB system based on a client verification of the QA pair.

Natural language interface databases

Systems and methods are provided for generating nested queries from natural language queries. In particular, system and methods are provided to implement natural language interfaces to databases (NLIDB) frameworks which are configured to apply intelligent reasoning over domain semantics to detect and generate nested queries across different domains without the need for domain specific training or utilizing domain-specific semantic templates for mapping a natural language query to a structured query.

Generating nested database queries from natural language queries

Methods, systems, and computer program products for automated generation of test cases for analyzing natural-language-interface-to-database systems are provided herein. A computer-implemented method includes identifying sources of ambiguity from input to a natural-language-interface-to-database system and a precondition corresponding to each of the identified sources of ambiguity; generating test cases which analyze capabilities of the natural-language-interface-to-database system, wherein generating the one or more test cases includes determining validity of the preconditions within the context of the capabilities of the natural-language-interface-to-database system; automatically generating an ontology-dependent structured query template based at least in part on the generated test cases; automatically generating natural language queries based at least in part on the ontology-dependent structured query template; and outputting, to at least one user, the ontology-dependent structured query template and the natural language queries.

Growing community of inventors

Bengaluru, India

Jaydeep Sen