Nuance Communications, Inc.

The Affinity Project, Inc.

Speechworks International, Inc.

Mark Fanty

John N Nguyen

Stephen Richard Springer

Ronald Steven Suskind

Stuart R Patterson

Michael Stuart Phillips

Krishna K Govindarajan

Michael Jack Newman

Michael D Edgington

Simona Gandrabur

Timothy Lynch

David Mark Krowitz

Anthony Gillet

Solomon Z Lerner

Etienne Barnard

Erik Martin Gregory

The method comprises receive first audio comprising speech from a user of a computing device, detecting an end of speech in the first audio, generating an ASR result based, at least in part, on a portion of the first audio prior to the detected end of speech, determining whether a valid action can be performed by a speech-enabled application installed on the computing device using the ASR result, and processing second audio when it is determined that a valid action cannot be performed by the speech-enabled application using the ASR result.

Methods and apparatus for reducing latency in speech recognition applications

An intelligent assistant can be used to facilitate an end user's participation in a virtual meeting. The intelligent assistant can receive recognized speech from a speech recognition engine, and then forward the recognized speech to a natural language understanding engine which in turn ascribes a meaning to the recognized speech. The ascribed meaning can include a semantic representation of the recognized speech and thereby a semantic representation of voice commands enshrined within the recognized speech. Using the ascribed meaning, the intelligent assistant then searches for meeting information associated with the virtual meeting and uses that meeting information to join the end user to the virtual meeting. Meeting information can include a conference call number, a participant code, a moderator code or a link to an online meeting.

Facilitating participation in a virtual meeting using an intelligent assistant

The operation of an application on a first device may be guided by a user operating a second device. The application on the first device may present a character on a display of the first device and obtain an audio signal of speech of a user of the first device. Audio data may be transmitted to the second device and corresponding audio may be played from speakers of the second device. The second device may present suggestions of phrases to be spoken by the character displayed on the first device. A user of the second device may select a phrase to be spoken by the character. Phrase data may be transmitted to the first device, and the first device may generate audio of the character speaking the phrase using a text-to-speech voice associated with the character.

Guided personal companion

A service may be provided that allows interacting users to interact with characters presented by devices of the interacting users. The operation of the character presented to an interacting user may be customized for that interacting user. For example, a first controlling user may specify first information for configuring or controlling the operation of the character when the character is interacting with a first interacting user, and a second controlling user may specify second information for configuring or controlling the operation of the character when the character is interacting with a second interacting user. The information provided by a controlling user may be used to select a segment to be used for a session between an interacting user and the character.

Configuring a virtual companion

A first user of a first device may interact with a character presented on the first device where the character presented on the first device is guided by a second user of a second device. To begin a communications session, the first user may request to communicate with the character. A server may process the request to speak with the character and identify the second user from a plurality of guiding users, where the second user is identified by comparing profile information of the first user with profile information of the second user. The server may then send a request to the second device of the second user to guide the character presented by the first device. The first user may then communicate with the character with the guidance of the second user.

On demand guided virtual companion

A user of a device may interact with a character presented by the device. A session between the user and the character may be started by a user, may be triggered by an event, or may be started in other ways. The user of the of the device may be associated with one or more trigger events. Information about the trigger events may be obtained and processed to determine when a trigger event has occurred. When a trigger event has occurred, a session may be started on the device with the character. A segment may be selected to be presented during the session, and an interaction corresponding to the segment may be presented by the character to the user. For example, the character may speak a phrase using a text-to-speech voice.

Triggering a session with a virtual companion

Methods and apparatus for reducing latency in speech recognition applications. The method comprises receive first audio comprising speech from a user of a computing device, detecting an end of speech in the first audio, generating an ASR result based, at least in part, on a portion of the first audio prior to the detected end of speech, determining whether a valid action can be performed by a speech-enabled application installed on the computing device using the ASR result, and processing second audio when it is determined that a valid action cannot be performed by the speech-enabled application using the ASR result.

Techniques for combining the results of multiple recognizers in a distributed speech recognition architecture. Speech data input to a client device is encoded and processed both locally and remotely by different recognizers configured to be proficient at different speech recognition tasks. The client/server architecture is configurable to enable network providers to specify a policy directed to a trade-off between reducing recognition latency perceived by a user and usage of network resources. The results of the local and remote speech recognition engines are combined based, at least in part, on logic stored by one or more components of the client/server architecture. An indication of the availability of the remote speech recognition to perform speech recognition at a point in time may be provided to a user of the client device via a user interface of the client device.

Configurable speech recognition system using multiple recognizers

In one embodiment, a method comprises classifying a representation of audio data of a dialog turn in a dialog system to a classification. The method may further comprise taking a security action on the classified representation of the audio data of the dialog turn as a function of the classification. The security action can be suppressing the representation of the audio data, encrypting the representation of the audio data, releasing the representation of the audio data, partially suppressing the representation of the audio data, partially encrypting the representation of the audio data, partially releasing the representation of the audio data, or a command.

Parsimonious protection of sensitive data in enterprise dialog systems

A telephone-based interactive speech recognition system is retrained using variable weighting and incremental retraining. Variable weighting involves changing the relative influence of particular measurement data to be reflected in a statistical model. Statistical model data is determined based upon an initial set of measurement data determined from an initial set of speech utterances. When new statistical model data is to be generated to reflect new measurement data determined from new speech utterances, a weighting factor is applied to the new measurement data to generate weighted new measurement data. The new statistical model data is then determined based upon the initial set of measurement data and the weighted new measurement data. Incremental retraining involves generating new statistical model data using prior statistical model data to reduce the amount of prior measurement data that must be maintained and processed. When prior statistical model data needs to be updated to reflect characteristics and attributes of new speech utterances, statistical model data is generated for the new speech utterances. Then the prior statistical model data and the statistical model data for the new measurement data are processed to generate the new statistical model data.

Automatically retraining a speech recognition system

Phonetic units are identified in a body of utterance data according to a novel segmentation approach. A body of received utterance data is processed and a set of candidate phonetic unit boundaries is determined that defines a set of candidate phonetic units. The set of candidate phonetic unit boundaries is determined based upon changes in Cepstral coefficient values, changes in utterance energy, changes in phonetic classification, broad category analysis (retroflex, back vowels, front vowels) and sonorant onset detection. The set of candidate phonetic unit boundaries is filtered by priority and proximity to other candidate phonetic units and by silence regions. The set of candidate phonetic units is filtered using no-cross region analysis to generate a set of filtered candidate phonetic units. No-cross region analysis generally involves discarding candidate phonetic units that completely span an energy up, energy down, dip or broad category type no-cross region. Finally, a set of phonetic units is selected from the set of filtered candidate phonetic units based upon the probabilities of candidate boundaries defining the ends of the unit and within the unit.

Segmentation approach for speech recognition systems

A method and apparatus are provided for improving the performance of an interactive speech application. The interactive speech application is developed and deployed for use by one or more callers. During execution, the interactive speech application stores, in a log, event information that describes each task carried out by the interactive speech application in response to interaction with the one or more callers. The application also stores one or more sets of audio information, in which each of the sets of audio information is associated with one or more utterances by one of the callers. Each of the sets of audio information is associated with one of the tasks represented in the log. After the log is established, an analytical report is displayed. The report describes selective actions taken by the interactive speech application while executing, and selective actions taken by one or more callers while interacting with the interactive speech application. Information in the analytical report is selected so as to identify one or more potential performance problems in the interactive speech application. While the analytical report is displayed, when the analytical report reaches a point at which the audio information was previously recorded and stored, the audio information may be replayed and analyzed. The interactive speech application is modified based on the analytical report. Accordingly, the interactive speech application may be improved based upon its actual performance, and its actual performance may be evaluated in detail based on specific call events and caller responses to application actions.

Method and system of reviewing the behavior of an interactive speech recognition application

An approach for automatically modifying a pronunciation dictionary in a speech recognition system based on patterns of alternate pronunciations is described. A representation of the pronunciation dictionary, such as a plurality of dynamically linked phoneme values, is obtained. One or more pattern definitions are obtained. The pattern definitions specify zero or more phonemes to be substituted for zero or more phonemes of all words in the pronunciation dictionary. The linked phoneme values are modified by adding, for each path of each word, alternate paths that use each of the substitute phonemes according to the pattern definitions, thereby creating an expanded set of dynamically linked phoneme values. One or more example pronunciations of a particular word are then obtained. One or more best paths through the expanded set of phoneme values are determined for each of the example pronunciations and used to find the overall best path(s). For the overall best path(s), an alternate word pronunciation is constructed by converting each path into a pronunciation using the format of the pronunciation dictionary. The pronunciation dictionary is modified by adding each alternate word pronunciation. As a result, a modified pronunciation dictionary is created that accounts for alternate pronunciations as actually spoken by users of a particular speech recognition application.

Method and apparatus for improved speech recognition by modifying a pronunciation dictionary based on pattern definitions of alternate word pronunciations

Growing community of inventors

Norfolk, MA, United States of America

Mark Fanty