Amazon Technologies, Inc.

Philip Ryan Hilmes

Robert Ayrapetian

Michael Alan Pogue

Trausti Thor Kristjansson

Amit Singh Chhetri

Kavitha Velusamy

Tony David

Ramya Gopalan

Preethi Narayanan

Carlo Murgia

Jun Yang

Yuwen Su

Jeffrey C O'Neill

Mohamed Farouk Mansour

Wai Chung Chu

William Folwell Barton

Shiva Kumar Sundaram

Guangji Shi

Chad Georges Seguin

Xianxian Zhang

Xiaoxue Li

Jan Aage Abildgaard Pedersen

Edward Dietz Crump

Shiv Naga Prasad Vitaladevuni

Colter Earl Cederlof

Rui Wang

Charles Guest Tritschler

Guangdong Pan

Hyeong Cheol Kim

Colin Randall McEnroe

Steve Anthony Quento

Steve Gil Gonzalez

Robert Gregory Deacon

A distributed voice controlled system has a primary assistant and at least one secondary assistant. The primary assistant has a housing to hold one or more microphones, one or more speakers, and various computing components. The secondary assistant is similar in structure, but is void of speakers. The voice controlled assistants perform transactions and other functions primarily based on verbal interactions with a user. The assistants within the system are coordinated and synchronized to perform acoustic echo cancellation, selection of a best audio input from among the assistants, and distributed processing.

Voice controlled system

Techniques for improving adaptive interference cancellation (AIC) using cascaded AIC algorithms are described. To improve an accuracy of detecting speech, a device may perform a first stage of AIC to generate isolated audio data and may generate speech mask data indicating time windows when speech is detected in the isolated audio data. Based on the speech mask data, the device may perform second AIC to generate output audio data, with adaptation of the adaptive filter enabled when the speech is not detected and disabled when the speech is detected. Thus, the first AIC improves the accuracy with which the device detects that speech is present and the second AIC reduces distortion in the output audio data by not updating filter coefficient values when the speech is present. The first AIC may use playback audio data, microphone audio data or beamformed audio data as reference signals.

Cascaded adaptive interference cancellation algorithms

A speech-based audio device may be configured to detect a user-uttered wake expression. For example, the audio device may generate a parameter indicating whether output audio is currently being produced by an audio speaker, whether the output audio contains speech, whether the output audio contains a predefined expression, loudness of the output audio, loudness of input audio, and/or an echo characteristic. Based on the parameter, the audio device may determine whether an occurrence of the predefined expression in the input audio is a result of an utterance of the predefined expression by a user.

Detecting self-generated wake expressions

Techniques for improving beamforming using filter coefficient values corresponding to virtual microphones are described. A system may define 'virtual' microphone positions and determine corresponding filter coefficient values. These filter coefficient values may be applied to input audio data captured by actual physical microphones, enabling the system to improve performance of beamforming and/or to reduce a number of physical microphones without degrading performance. Offline testing and simulations may be performed to identify the best combination of virtual microphones and/or filter coefficient values for a particular look-direction. For example, the simulations may identify that a first filter coefficient corresponding to a first virtual microphone and a first direction will be associated with a first physical microphone and the first direction. During run-time processing, a device may generate beamformed audio data for the first direction by applying the first filter coefficient to input audio data captured by the first physical microphone.

Beamforming using filter coefficients corresponding to virtual microphones

Systems and methods for microphone degradation detection and compensation are disclosed. For example, microphones of an electronic device may capture audio and generate corresponding audio data, such as during a period of time where only ambient noise is present. Sound intensity level value differences between audio data from the various microphones may be determined and when one or more of the sound intensity level value differences satisfies a threshold amount, the microphone associated with the variant sound intensity level value may be determined to be degraded. The sound intensity level value difference may be compensated for, such as by utilizing sound boosting techniques and/or modifying parameters of a beamforming component.

Microphone degradation detection and compensation

A system that performs wall detection, range estimation, and/or corner detection to determine a position of a device relative to acoustically reflective surfaces. The device generates output audio using a loudspeaker, generates microphone audio data using a microphone array, performs beamforming to generate directional audio data and then generates impulse response data for each of a plurality of directions. The device may detect a peak in the impulse response data and determine a distance and/or direction to a reflective surface based on the peak. Based on a number of reflected surfaces and/or direction(s) of the reflected surfaces detected by the device, the device may classify the different directions and estimate where it is in the room, such as whether the device is in a corner, along one wall, or in an open area. By knowing its position relative to the room surfaces, the device may improve sound equalization.

Method for autonomous loudspeaker room adaptation

A system configured to improve audio processing by adaptively selecting target signals based on current system conditions. For example, a device may select a target signal based on a highest signal quality metric when only the local speech is present (e.g., during near-end single-talk conditions), as this maximizes an amount of energy included in the output audio signal. In contrast, the device may select the target signal based on a lowest signal quality metric when only the remote speech is present (e.g., during far-end single-talk conditions), as this minimizes an amount of energy included in the output audio signal. In addition, the device may track positions of the local speech and the remote speech over time, enabling the device to accurately select the target signal when both local speech and remote speech is present (e.g., during double-talk conditions).

Beam level based adaptive target selection

A wearable computer is configured to use beamforming techniques to isolate a user's speech from extraneous audio signals occurring within a physical environment. A microphone array of the wearable computer may generate audio signal data from an utterance from a user's mouth. A motion sensor(s) of the wearable computer may generate motion data from movement of the wearable computer. This motion data may be used to determine a direction vector pointing from the wearable computer to the user's mouth, and a beampattern may be defined that has a beampattern direction in substantial alignment with the determined direction vector to focus the microphone array on the user's mouth for speech isolation.

Beamforming for a wearable computer

A system that performs wall detection, range estimation, and/or corner detection to determine a position of a device relative to acoustically reflective surfaces. The device generates output audio using loudspeaker(s), generates microphone audio data using a microphone array, and generates impulse response data for each of the microphones. The device may generate the impulse response data using an acoustic echo cancellation (AEC) component or multi-channel AEC (MC-AEC). The device may detect a peak in the impulse response data and determine a distance to a reflective surface based on the peak. Based on a number of reflected surfaces detected by the device, the device may classify a position of the device within the room, such as whether the device is in a corner, along one wall, or in an open area. By knowing the position relative to the room surfaces, the device may improve sound equalization and other processing.

Method for efficient autonomous loudspeaker room adaptation

A system configured to improve noise cancellation by using portions of multiple reference signals instead of using a complete reference signal. The system divides a frequency spectrum into frequency bands and selects a single reference signal from a group of potential reference signals for every frequency band. For example, a first reference signal is selected for a first frequency band while a second reference signal is selected for a second frequency band. The system may generate a combined reference signal using portions of each of the selected reference signals, such as a portion of the first reference signal corresponding to the first frequency band and a portion of the second reference signal corresponding to the second frequency band. Additionally or alternatively, the system may perform noise cancellation using each of the selected reference signals and filter the outputs based on the corresponding frequency band to generate combined audio output data.

Multichannel noise cancellation using frequency domain spectrum masking

Growing community of inventors

San Jose, CA, United States of America

Philip Ryan Hilmes