International Business Machines Corporation

Nuance Communications, Inc.

Rajesh Balchandran

Linda M Boyer

Gregory Purdy

James Robert Lewis

Brent D Metz

Ilya Skuratovsky

A method () and system () is provided to add the creation of examples at a developer level in the generation of Natural Language Understanding (NLU) models, tying the examples into a NLU sentence database (), automatically validating () a correct outcome of using the examples, and automatically resolving () problems the user has using the examples. The method () can convey examples of what a caller can say to a Natural Language Understanding (NLU) application. The method includes entering at least one example associated with an existing routing destination, and ensuring an NLU model correctly interprets the example unambiguously for correctly routing a call to the routing destination. The method can include presenting the example sentence in a help message () within an NLU dialogue as an example of what a caller can say for connecting the caller to a desired routing destination. The method can also include presented a failure dialogue for displaying at least one example that failed to be properly interpreted to ensure that ambiguous or incorrect examples are not presented in a help message.

Method and system for conveying an example in a natural language understanding application

A method of extracting information from text within a natural language understanding system can include processing a text input through at least one statistical model for each of a plurality of features to be extracted from the text input. For each feature, at least one value can be determined, at least in part, using the statistical model associated with the feature. One value for each feature can be combined to create a complex information target. The complex information target can be output.

Information extraction in a natural language understanding system

A method () and system () is provided to add the creation of examples at a developer level in the generation of Natural Language Understanding (NLU) models, tying the examples into a NLU sentence database (), automatically validating () a correct outcome of using the examples, and automatically resolving () problems the user has using the examples. The method () can convey examples of what a caller can say to a Natural Language Understanding (NLU) application. The method includes entering at least one example associated with an existing routing destination, and ensuring an NLU model correctly interprets the example unambiguously for correctly routing a can to the routing destination. The method can include presenting the example sentence in a help message () within an NLU dialog as an example of what a caller can say for connecting the caller to a desired routing destination. The method can also include presented a failure dialog for displaying at least one example that failed to be properly interpreted to ensure that ambiguous or incorrect examples are not presented in a help message.

A method of creating a statistical classification model for a classifier within a natural language understanding system can include processing training data using an existing statistical classification model. Sentences of the training data correctly classified into a selected class of the statistical classification model can be selected. The selected sentences of the training data can be assigned to a fringe group or a core group according to confidence score. The training data can be updated by associating the fringe group with a fringe subclass of the selected class and the core group with a core subclass of the selected class. A new statistical classification model can be built from the updated training data. The new statistical classification model can be output.

Reclassification of training data to improve classifier accuracy

A method for collecting data for statistical modeling purposes can include the step of selecting at least one user interface type from a plurality of previously defined user interface types. Parameters of the selected interface type can be defined for a particular data collection instance. Target participant data can be inputted. A data collection interface based upon the selected interface type and defined parameters can be deployed. Messages can be automatically conveyed to data providers selected in accordance with the target participant data. The data providers can be permitted to access the deployed data collection interface. Data provided by the data providers can be automatically stored and used for statistical modeling purposes related to the data collection instance.

System and method for data collection interface creation and data collection administration

A method of classifying text input for use with a natural language understanding system can include determining classification information including a primary classification and one or more secondary classifications for a received text input using a statistical classification model (statistical model). A statistical classification sub-model (statistical sub-model) can be selectively built according to a model generation criterion applied to the classification information. The method further can include selecting the primary classification or the secondary classification for the text input as a final classification according to the statistical sub-model and outputting the final classification for the text input.

Sub-model generation to improve classification accuracy

A method for processing language input can include the step of determining at least two possible meanings for a language input. For each possible meaning, a probability that the possible meaning is a correct interpretation of the language input can be determined. At least one relative data computation can be computed based at least in part upon the probabilities. At least one irregularity within the language input can be detected based upon the relative delta computation. The irregularity can include mumble, ambiguous input, and/or compound input. At least one programmatic action can be performed responsive to the detection of the irregularity.

Relative delta computations for determining the meaning of language inputs

A method of processing text within a natural language understanding system can include applying a first tokenization technique to a sentence using a statistical tokenization model. A second tokenization technique using a named entity can be applied to the sentence when the first tokenization technique does not extract a needed token according to a class of the sentence. A token determined according to at least one of the tokenization techniques can be output.

Extracting tokens in a natural language understanding application

In a natural language, mixed-initiative system, a method of processing user dialogue can include receiving a user input and determining whether the user input specifies an action to be performed or a token of an action. The user input can be selectively routed to an action interpreter or a token interpreter according to the determining step.

Method, system, and apparatus for natural language mixed-initiative dialogue processing

The invention disclosed herein concerns a system () and method () for building a language model representation of an NLU application. The methodcan include categorizing an NLU application domain (), classifying a corpus in view of the categorization (), and training at least one language model in view of the classification (). The categorization produces a hierarchical tree of categories, sub-categories and end targets across one or more features for interpreting one or more natural language input requests. During development of an NLU application, a developer assigns sentences of the NLU application to categories, sub-categories or end targets across one or more features for associating each sentence with desire interpretations. A language model builder () iteratively builds multiple language models for this sentence data, and iteratively evaluating them against a test corpus, partitioning the data based on the categorization and rebuilding models, so as to produce an optimal configuration of language models to interpret and respond to language input requests for the NLU application.

Method and system for automatically building natural language understanding models

A method for identifying data that is meaningless and generating a natural language statistical model which can reject meaningless input. The method can include identifying unigrams that are individually meaningless from a set of training data. At least a portion of the unigrams identified as being meaningless can be assigned to a first n-gram class. The method also can include identifying bigrams that are entirely composed of meaningless unigrams and determining whether the identified bigrams are individually meaningless. At least a portion of the bigrams identified as being individually meaningless can be assigned to the first n-gram class.

Growing community of inventors

Congers, NY, United States of America

Rajesh Balchandran