Amazon Technologies, Inc.

Canyon IP Holdings, LLC

Kenneth John Basye

Jeffrey Penrod Adams

Hugh Evan Secker-Walker

Mahesh Krishnamoorthy

Rohit Prasad

Bjorn Hoffmeister

Spyridon Matsoukas

Shiv Naga Prasad Vitaladevuni

Tony David

Stan Weidner Salvador

Ryan Paul Thomas

Rajiv Ramachandran

Reinhard Kneser

Alexander David Rosen

Nikko Strom

William Tunstall-Pedoe

Jeffrey C O'Neill

Jake Kramer

Michael Maximilian Emanuel Bisani

A speech recognition system utilizing automatic speech recognition techniques such as end-pointing techniques in conjunction with beamforming and/or signal processing to isolate speech from one or more speaking users from multiple received audio signals and to detect the beginning and/or end of the speech based at least in part on the isolation. Audio capture devices such as microphones may be arranged in a beamforming array to receive the multiple audio signals. Multiple audio sources including speech may be identified in different beams and processed.

Direction based end-pointing for speech recognition

Features are disclosed for detecting words in audio using contextual information in addition to automatic speech recognition results. A detection model can be generated and used to determine whether a particular word, such as a keyword or 'wake word,' has been uttered. The detection model can operate on features derived from an audio signal, contextual information associated with generation of the audio signal, and the like. In some embodiments, the detection model can be customized for particular users or groups of users based usage patterns associated with the users.

Wake word detection modeling

Power consumption for a computing device may be managed by one or more keywords. For example, if an audio input obtained by the computing device includes a keyword, a network interface module and/or an application processing module of the computing device may be activated. The audio input may then be transmitted via the network interface module to a remote computing device, such as a speech recognition server. Alternately, the computing device may be provided with a speech recognition engine configured to process the audio input for on-device speech recognition.

Speech recognition power management

Keyword detection modeling using contextual information

An efficient audio streaming method and apparatus includes a client process implemented on a client or local device and a server process implemented on a remote server or server(s). The client process and server process each have speech recognition components and communicate over a network, and together efficiently manage the detection of speech in an audio signal streamed by the local device to the server for speech recognition and potentially further processing at the server. The client process monitors audio input and in a first detection stage, implements endpointing on the local device to determine when speech is detected. The client process may further determine if a 'wakeword' is detected, and then the client process opens a connection and begins streaming audio to the server process via the network. The server process receives the speech audio stream and monitors the audio, implementing endpointing in the server process, to determine when to tell the client process to close the connection and stop streaming audio. The client process continues streaming audio to the server until the server process determines disconnect criteria have been met and tells the client process to stop streaming audio.

Distributed endpointing for speech recognition

Features are disclosed for detecting words in audio using environmental information and/or contextual information in addition to acoustic features associated with the words to be detected. A detection model can be generated and used to determine whether a particular word, such as a keyword or 'wake word,' has been uttered. The detection model can operate on features derived from an audio signal, contextual information associated with generation of the audio signal, and the like. In some embodiments, the detection model can be customized for particular users or groups of users based usage patterns associated with the users.

Keyword detection modeling using contextual and environmental information

In a dynamic automatic speech recognition (ASR) processing system, ASR processing may be configured to estimate a latency of returning speech results to a user based on work being done by an ASR processor. The ASR processing system may measure work done by an ASR processor by measuring one or more time independent metrics and comparing the metrics to threshold values. If the metrics exceed the thresholds, the ASR system may take steps to reduce latency associated with processing the utterance, including adjusting a speech recognition parameter.

Dynamic pruning in speech recognition

Devices, methods, and systems for detecting wake words and audio commands that should be disregarded are disclosed. In some instances, a local device may receive a wake word or audible command transmitted or uttered in a television or radio advertisement, program, broadcast, etc. In these instances, the local device should disregard such wake words and audible commands, as they are not from a user of the local device. To detect such wake words and commands, audio fingerprinting and speech recognition techniques may be used to determine whether the wake word and/or command substantially matches the audio of a known television or radio advertisement, program, broadcast, etc. If the wake word and/or command substantially matches, the local device may then disregard the command.

Audible command filtering

In an automatic speech recognition (ASR) processing system, ASR processing may be configured to reduce a latency of returning speech results to a user. The latency may be determined by comparing a time stamp of an utterance in process to a current time. Latency may also be estimated based on an endpoint of the utterance or other considerations such as how difficult the utterance may be to process. To improve latency the ASR system may be configured to adjust various processing parameters, such as graph pruning factors, path weights, ASR models, etc. Latency checks and corrections may occur dynamically for a particular utterance while it is being processed, thus allowing the ASR system to adjust to rapidly changing latency conditions.

Growing community of inventors

Sutton, MA, United States of America

Kenneth John Basye