International Business Machines Corporation

Ashish Mittal

Diptikalyan Saha

Karthik Sankaranarayanan

Jaydeep Sen

Samarth Bharadwaj

Shreya Khare

Manasa Jammi

Vijay Ekambaram

Ruhi Sharma Mittal

Yedendra Shrinivasan

Priyanka Agrawal

Parag Jain

Saneem Ahmed Chemmengath

Yedendra B Shrinivasan

A method, system, and computer program product for automated artificial intelligence (AI) factsheet generation for modeling and model customization in speech to text (STT) services. The method receives audio data for a user. The audio data contains human speech. Text data is generated, using a first speech to text model, to represent the human speech of the audio data. A set of transcription errors of the first speech to text model are identified. A set of AI factsheets are generated to describe model metadata for the first speech to text model. Based on the set of transcription errors and the set of AI factsheets, the method generates a second speech to text model customized to the user.

Artificial intelligence factsheet generation for speech recognition

An automatic speech recognition (ASR) computing system and methodology are provided to predict a textual representation of received input speech data. A context acoustic biasing (CAB) engine of the ASR computing system receives historical textual content and an ontology data structure. The CAB engine matches key terms identified in the historical textual content with concepts present in the ontology data structure to generate a contextual term list data structure comprising the concept terms related to concepts matching the key terms. The CAB engine generates acoustic representations of the concept terms in the contextual term list data structure and inputs them to an ASR computer model of the ASR computing system which processes an input speech signal to generate a predicted textual representation of the input speech signal. The predicted textual representation is biased towards the acoustic representations of the concept terms in the contextual term list data structure.

Ontology driven contextual automated speech recognition

Methods, systems, and computer program products for automated domain-specific constrained decoding from speech inputs to structured resources are provided herein. A computer-implemented method includes converting at least a portion of at least one user-provided speech utterance into text by processing the at least one user-provided speech utterance using an artificial intelligence-based automatic speech recognition model; automatically training an artificial intelligence-based decoding engine, wherein automatically training the artificial intelligence-based decoding engine comprising constraining the artificial intelligence-based decoding engine based at least in part on a domain-specific model and the artificial intelligence-based automatic speech recognition model; and generating at least one of one or more domain-specific text outputs related to one or more structured resources associated with the domain and one or more domain-specific action outputs related to the one or more structured resources associated with the domain by processing at least a portion of the text using the artificial intelligence-based decoding engine.

Automated domain-specific constrained decoding from speech inputs to structured resources

One embodiment provides a method, including: receiving, within a conversational agent creation framework used to create conversational agents that perform negotiations on behalf of users, (i) a selection of one of a plurality of conversational agent roles (ii) negotiation constraints for the selected role, and (iii) an object for the negotiation; creating a conversational agent (i) having the selected role and (ii) programmed with the negotiation constraints; identifying at least one other conversational agent (i) having an opposing role with respect to the object and (ii) being programmed with opposing role negotiation constraints; and conducting a conversational session between the conversational agent and the at least one other conversational agent, wherein the conversational session comprises the negotiation for the object and wherein the conversational agent and the at least one other conversational agent perform the negotiation in view of the negotiation constraints and the opposing role negotiation constraints.

Multi-agent conversational agent framework

Methods, systems, and computer program products for active learning for natural language question answering are provided herein. A computer-implemented method includes generating a semantic signature of a natural language query; generating a SQL signature for a SQL query corresponding to the natural language query; determining whether a set of mappings includes a semantic signature matching the generated semantic signature, wherein each mapping in the set is between (i) a given semantic signature and (ii) a SQL signature representing a class of SQL queries corresponding to the given semantic signature; and in response to determining that the set of mappings does not include the generated semantic signature, adding a mapping between the generated semantic signature and the generated SQL signature to the set of mappings.

Active learning for natural language question answering

One embodiment provides a method, including: receiving a query from a user; identifying that a desired definition of the at least one term is unknown, by determining that the at least one term does not map to a term having a known definition; receiving the definition of the at least one term from the user; adding the definition to a domain grammar comprising (i) domain-specific terminology and (ii) definitions corresponding to the terms within the domain grammar, wherein the adding comprises (a) extracting expressions from the requested definition and (b) adding, for the at least one term, the expressions into a structured format within the domain grammar; combining (iii) the requested definition and (iv) terms from the parsed query having previously known definitions into a complete query; and providing a response to the query by executing the complete query on a knowledge store.

Domain query execution using user-provided definition

A computer-implemented wardrobe packing method, system, and a computer program product, include selecting a plurality of items stored in a memory to pack into a container, generating a packing order of the plurality of items based on packing preference data and item data stored in the memory, creating and displaying a video of step-by-step packing instructions for the plurality of items in the packing order, monitoring and comparing an actual packing process to a generated packing order, and controlling the displaying of a step of the step-by-step packing instructions, in response to the monitoring and comparing.

Providing notifications in video streams to control video playback

Embodiments are disclosed for correcting a natural language interface database (NLIDB) system. The techniques include receiving feedback indicating that an answer provided in response to a question for an NLIDB system is inaccurate. The techniques further include finding an ontology element for a datastore of the NLIDB system that matches to the feedback. The techniques also include selecting candidate annotations for the NLIDB system based on the ontology element and a data type of the ontology element. Additionally, the techniques include generating a question-answer (QA) pair for each of the candidate annotations. Further, the techniques include adding one of the candidate annotations to annotations for a natural language query (NLQ) engine of the NLIDB system based on a client verification of the QA pair.

Natural language interface databases

Methods, systems, and computer program products for automated generation of test cases for analyzing natural-language-interface-to-database systems are provided herein. A computer-implemented method includes identifying sources of ambiguity from input to a natural-language-interface-to-database system and a precondition corresponding to each of the identified sources of ambiguity; generating test cases which analyze capabilities of the natural-language-interface-to-database system, wherein generating the one or more test cases includes determining validity of the preconditions within the context of the capabilities of the natural-language-interface-to-database system; automatically generating an ontology-dependent structured query template based at least in part on the generated test cases; automatically generating natural language queries based at least in part on the ontology-dependent structured query template; and outputting, to at least one user, the ontology-dependent structured query template and the natural language queries.

Automated generation of test cases for analyzing natural-language-interface-to-database systems

Methods, systems, and computer program products for bootstrapping of state-based dialog systems are provided herein. A computer-implemented method includes determining parameters for state automata by partitioning an ontology graph into sub-graphs and a knowledge graph into sub-graphs, wherein the ontology graph and the knowledge graph are based on a user question and domain knowledge pertaining to the user question; generating a structured query for each of the sub-graphs; determining intentions of a dialog pertaining to the at least one user question by translating each of the generated structured queries to a respective natural language query; creating one or more dialog states for each of the determined dialog intentions; creating one or more connecting dialog states between pairs of the created dialog states; and generating an automata dialog framework associated with the user question based on the created dialog states and the created connecting dialog states.

Ontology-based automatic bootstrapping of state-based dialog systems

Aspects of the present disclosure relate to automated ontology refinement based on query inputs and provided feedback. A query input is received for an ontology. Features of the query input are analyzed, wherein analyzation includes determining syntactical and semantic characteristics of the features of the query input. Based on the determined syntactical and semantic characteristics, ontological elements are classified for each feature of the query input. The ontological element for each feature of the query input is then compared to a set of ontological elements of the ontology. Based on the comparison, a response to the query input is received, along with a request for feedback regarding the response. Feedback is then received regarding the response. Based on the feedback, the ontology is analyzed to determine at least one deficiency of the ontology. The ontology is then refined to correct the at least one deficiency.

Ontology refinement based on query inputs

Methods, systems, and computer program products for carrying out interactive dialog in natural language using an ontology are provided herein. A computer-implemented method includes generating one or more domain-driven interpretations of a natural language dialogue query provided by a user via utilization of a web ontology language; determining multiple structured base queries, from among a stored collection of structured queries, that correspond to the natural language dialogue query, in view of the one or more generated domain-driven interpretations; selecting one of the multiple determined structured base queries, based on one or more items of context information pertaining to the dialogue; automatically generating a response to the selected structured base query; and outputting the generated response to the user.

Interactive dialog in natural language using an ontology

A computer-implemented wardrobe packing method, system, and a computer program product, include selecting a plurality of items stored in a memory to pack into a container, generating a packing order of the plurality of items based on packing preference data and item data stored in the memory, creating and displaying a video of step-by-step packing instructions for the plurality of items in the packing order, monitoring and comparing an actual packing process to a generated packing order, and controlling said displaying of step of the step-by-step packing instructions, in response to said monitoring and comparing.

System, method and computer program product for packing

Methods, systems, and computer program products for translating structured languages to natural language using domain-specific ontology are provided herein. A computer-implemented method includes determining similarities among multiple natural language query interpretations derived from an input query, determining differences among the multiple natural language query interpretations, and generating natural language descriptions of each of the multiple natural language query interpretations based on analysis of the determined similarities, the determined differences, and the input query. The method also includes producing, for each of the natural language query interpretations, a natural language string that represents one or more unambiguous interpretations of the input query, wherein the producing comprises consolidating the generated natural language descriptions. Further, the method includes outputting each of the produced natural language strings to a user.

Growing community of inventors

Bengaluru, India

Ashish Mittal