Nuance Communications, Inc.

Sri International

Michael H Cohen

Eric G Jackson

Tracy D Wax

Hy Murveit

Mitchel Weintraub

Jared C Bernstein

Patti J Price

Jennifer E Balogh

Larry Paul Heck

Horacio E Franco

Steven Clive Ehrlich

Debajit Ghosh

James M Riseman

Madhavan S Thirumalai

Naghmeh N Mirghafori

Michael A Prince

Vijaya L Kurada

A system which uses automatic speech recognition to provide dialogs with human speakers automatically detects one or more characteristics, which may be characteristics of a speaker, his speech, his environment, or the speech channel used to communicate with the speaker. The characteristic may be detected either during the dialog or at a later time based on stored data representing the dialog. If the characteristic is detected during the dialog, the dialog can be customized for the speaker at an application level, based on the detected characteristic. The customization may include customization of operations and features such as call routing, error recovery, call flow, content selection, system prompts, or system persona. Data indicative of detected characteristics can be stored and accumulated for many speakers and/or dialogs and analyzed offline to generate a demographic or other type of analysis of the speakers or dialogs with respect to one or more detected characteristics.

Detection of characteristics of human-machine interactions for dialog customization and analysis

The present invention allows a user to audibly and interactively browse through a network of audio information, forming a seamless integration of the world wide web and the entire telephone network browsable from any telephone set. Preferably a browser controller allows the user to receive audio information and to transmit verbal instructions. The browser controller links the user to voice pages, which can be any telephone station or world wide web page, in response to voice commands. Upon linking, certain information is played with an audio indicia which identifies a linking capability. If the user repeats the information set off by the audio indicia, the telephone number or URL of the selected link is transmitted to the browser controller. The browser controller establishes a new link with the identified telephone number or URL, and if successful, disconnects the previous link. The originator no longer needs to know of the existence of the receiver nor the telephone number or URL of the receiver because this invention provides a method to browse the entire telephone network and world wide web and to connect to a receiver by saying the name of the hyperlink. This brings the power of the world wide web to the telephone network. In effect, this invention takes the PSTN from its current state as a set of more than 800 million nodes including means to make pairwise connections and converts it to a highly interconnected browsable web, as well as integrating it with the entire world wide web.

System for and method of creating and browsing a voice web

A network comprises a number of speech-enabled sites maintaining a number of voice pages. A central server on the network executes a voice browser which provides users with access to the sites using voice-activated hyperlinks. The server also maintains and brokers information associated with the users based on spoken dialogs between the users and the sites. In response to a user accessing a given ASR site, information about that user is provided by the server for use by that ASR site. The information is used by the ASR site to optimize a spoken dialog between the user and the ASR site by reducing the amount of information the user is required to provide during the dialog. Information about the user can thereby be shared between separate speech enabled sites, in a manner which is transparent to the user, in order to expedite the user's interaction with those sites.

Method and apparatus for optimizing a spoken dialog between a person and a machine

A computerized method for building and running natural language understanding systems, wherein a natural language understanding system takes a sentence as input and returns some representation of the possible meanings of the sentence as output (the &ldquo;interpretation&rdquo;) using a run-time interpreter th assigns interpretations to sentences and a compiler that produces (in a computer memory) an internal specification needed for the run-time interpreter from a user specification of the semantics of the application. The compiler builds a natural language system, while the run-time interpreter runs the system.

Method and system for building and running natural language understanding systems

A voice-enabled application, which may be a voice browser, is configured to provide active help to a user. The application maintains a usage history of each user with respect to dialog states. The usage history includes various user-specific variables, some of which are valid across multiple sessions. The application maintains a number of active help prompts capable of being played to a user as speech, each containing information on a different, specific help topic. The application further maintains a number of sets of conditions, each set corresponding to a different one of the active help prompts. The application monitors dialog states during a session of a user and generates an event based on the dialog states. The application applies certain ones of the conditions to certain ones of the user-specific variables in response to the event. The application then plays a active help prompt containing information on a specific help topic to the user if the applied conditions are satisfied.

Method and apparatus for providing active help to a user of a voice-enabled application

A computerized method for building and running natural language understanding systems, wherein a natural language understanding system takes a sentence as input and returns some representation of the possible meanings of the sentence as output (the &ldquo;interpretation&rdquo;) using a run-time interpreter that assigns interpretations to sentences and a compiler that produces (in a computer memory) an internal specification needed for the run-time interpreter from a user specification of the semantics of the application. The compiler builds a natural language system, while the run-time interpreter runs the system.

A method and apparatus are used for analyzing dialogs in an interactive natural language speech recognition system. As a telephone call is received in the interactive natural language speech recognition system, user utterances are digitized and stored. A data log is formed which stores the prompt played, the dialog state, the address location of the audio recording of the user's responses, the recognition grammar, the recognition result type, the recognized word string, the understood meaning/interpretation, and the recognition latency. The telephone call can be virtually reconstructed from the stored user utterances and the data log. An automatic data analysis tool provides information regarding the relative success and failure of the various dialogs as an aid to modifying the dialogs to improve system performance.

Method of analyzing dialogs in a natural language speech recognition system

An automatic speech recognition methodology, wherein words are modeled as probabilistic networks of allophones, collects nodes in the probabilistic network into equivalence classes when those nodes have the same allophonic choices governed by the same phonological rules. The allophonic choices allow for representation of dialectic pronunciation variations between different speakers. Training data is shared among nodes in an equivalence class so that accurate pronunciation probabilities may be determined even for words for which there is only a limited amount of training data. A method is used to determine probabilities for each of a multitude of pronunciation models for each word in the vocabulary, based on automatic extraction of linguistic knowledge from sets of phonological rules, in order to robustly and accurately model dialectal variation.

Method for recognizing speech using linguistically-motivated hidden

In a hidden Markov model-based speech recognition system, multilayer perceptrons (MLPs) are used in context-dependent estimation of a plurality of state-dependent observation probability distributions of phonetic classes. Estimation is obtained by the Bayesian factorization of the observation likelihood in terms of posterior probabilities of phone classes assuming the context and the input speech vector. The context-dependent estimation is employed as the state-dependent observation probabilities needed as parameter input to a hidden Markov model speech processor to identify the word sequence representing the unknown speech input of input speech vectors. Within the speech processor, models are provided which employ the observation probabilities in the recognition process. The number of context-dependent nets is reduced to a single net by sharing the units of the input layer and the hidden layer and the weights connecting them in the multilayer perceptron while providing one output layer for each relevant context. Each output layer is trained as an independent network on the specific examples of the corresponding context it represents. Training may be optimized at an intermediate set of weights between the context-independent-associated weights and the context-dependent associated weights to which training would normally converge.

Method and apparatus for context-dependent estimation of multiple

An automatic speech recognition methodology takes advantage of linguistic constraints wherein words are modeled as probabilistic networks of phonetic segments (herein phones), and each phone is represented as a context-independent hidden Markov phone model mixed with a number of context-dependent phone models. Recognition is based on use of methods to design phonological rule sets based on measures of coverage and overgeneration of pronunciations which achieves high coverage of pronunciations with compact representations. Further, a method estimates probabilities of the different possible pronunciations of words. A further method models cross-word coarticulatory effects. In a specific embodiment of the system, a specific method determines the single most-likely pronunciation of words. In further specific embodiments of the system, methods generate speaker-dependent pronunciation networks.

Growing community of inventors

Burlingame, CA, United States of America

Michael H Cohen