Amazon Technologies, Inc.

Other

Ikanos Communications, Inc.

Marvell International Limited

Robert Ayrapetian

Philip Ryan Hilmes

Yuwen Su

Arnaud Jean-Louis Charton

Michael Alan Pogue

Carlo Murgia

Trausti Thor Kristjansson

Mohamed Farouk Mansour

Kurt W Piersol

Wai Chung Chu

Preethi Narayanan

Kenneth Edgar Hild, Ii

Pradeep Kumar Govindaraju

Gabe Beddingfield

Sam Heidari

Amit Singh Chhetri

Bjorn Hoffmeister

Ludger Solbach

Raphael Cendrillon

Krishna Kamath Koteshwara

Dale Smith

Qasem Aldrubi

Carlos Renato Nakagawa

Navin Chatlani

Hossein Dehghan-Fard

Guangdong Pan

Fernando Ramirez-Mireles

Hyeong Cheol Kim

John Walter McDonough, Jr

Christopher Chow

Max Chien

Gautam Shreedhar Bhat

Inseok Heo

Edward Ayrapetian

Serob Douvalian

Techniques for improving microphone noise suppression are provided. When a playback signal is available, a device may use the playback signal as a reference while performing Acoustic Echo Cancellation (AEC) processing. When the playback signal is not available, the device may instead use microphone audio signal(s) as the reference while performing Adaptive Interference Cancellation (AIC) processing. For example, the device may use first audio data generated by first microphones as the reference while performing AIC processing on the first audio data and second audio data generated by second microphones. As the adaptive filter used to perform AIC processing is updated at a slower rate, the reference signal only cancels stationary portions of the first audio data (e.g., noise), leaving a representation of transient sounds such as speech. In some examples, the device may perform AIC processing and AEC processing in series.

Microphone reference echo cancellation

A system that performs beam selection and beam merging using beam-specific signal quality metrics corresponding to a minimum noise floor. For example, a device may track a minimum noise floor for each beam, determine a highest minimum noise floor across the beams, and determine a noise floor ratio between the beam-specific minimum noise floor and the highest minimum noise floor. Using a combination of the noise floor ratio and signal-to-noise ratio (SNR) values, the device may perform beam selection by prioritizing low background noise as well as high SNR to select a pre-defined beam group. In addition, the device may use the noise floor ratio to perform beam merging and generate single-channel output audio data using the selected beam group. For example, the device may scale the beams based on a combination of the SNR value and the noise floor ratio.

Group beam selection and beam merging

Techniques for improving adaptive interference cancellation (AIC) using cascaded AIC algorithms are described. To improve an accuracy of detecting speech, a device may perform a first stage of AIC to generate isolated audio data and may generate speech mask data indicating time windows when speech is detected in the isolated audio data. Based on the speech mask data, the device may perform second AIC to generate output audio data, with adaptation of the adaptive filter enabled when the speech is not detected and disabled when the speech is detected. Thus, the first AIC improves the accuracy with which the device detects that speech is present and the second AIC reduces distortion in the output audio data by not updating filter coefficient values when the speech is present. The first AIC may use playback audio data, microphone audio data or beamformed audio data as reference signals.

Cascaded adaptive interference cancellation algorithms

Techniques for improving microphone noise suppression are provided. As wind noise may disproportionately impact a subset of microphones, a method for processing audio data using two adaptive reference algorithm (ARA) paths in parallel is provided. For example, first ARA processing performs noise cancellation using all microphones, while second ARA processing performs noise cancellation using only a portion of the microphones. As the first ARA processing and the second ARA processing are performed in parallel, beam merging can be performed using beams from the first ARA, the second ARA, and/or a combination of each. In addition, beam merging can be performed using beam sections instead of individual beams to further improve performance and reduce attenuation to speech.

Parallel noise suppression

Techniques for improving beamforming using filter coefficient values corresponding to virtual microphones are described. A system may define 'virtual' microphone positions and determine corresponding filter coefficient values. These filter coefficient values may be applied to input audio data captured by actual physical microphones, enabling the system to improve performance of beamforming and/or to reduce a number of physical microphones without degrading performance. Offline testing and simulations may be performed to identify the best combination of virtual microphones and/or filter coefficient values for a particular look-direction. For example, the simulations may identify that a first filter coefficient corresponding to a first virtual microphone and a first direction will be associated with a first physical microphone and the first direction. During run-time processing, a device may generate beamformed audio data for the first direction by applying the first filter coefficient to input audio data captured by the first physical microphone.

Beamforming using filter coefficients corresponding to virtual microphones

A system configured to improve echo cancellation for nonlinear systems. The system generate reference audio data by isolating portions of microphone audio data that correspond to playback audio data. For example, the system may determine a correlation between the playback audio data and the microphone audio data in individual time-frequency bands in a frequency domain. In some examples, the system may substitute microphone audio data associated with output audio for the playback audio data. The system may generate the reference audio data based on portions of the microphone audio data that have a strong correlation with the playback audio data. The system may generate the reference audio data by selecting these portions of the microphone audio data or by performing beamforming. This results in precise time alignment between the reference audio data and the microphone audio data, improving performance of the echo cancellation.

Echo cancellation by acoustic playback estimation

A system configured to improve noise cancellation by using portions of multiple reference signals instead of using a complete reference signal. The system divides a frequency spectrum into frequency bands and selects a single reference signal from a group of potential reference signals for every frequency band. For example, a first reference signal is selected for a first frequency band while a second reference signal is selected for a second frequency band. The system may generate a combined reference signal using portions of each of the selected reference signals, such as a portion of the first reference signal corresponding to the first frequency band and a portion of the second reference signal corresponding to the second frequency band. Additionally or alternatively, the system may perform noise cancellation using each of the selected reference signals and filter the outputs based on the corresponding frequency band to generate combined audio output data.

Multichannel noise cancellation using frequency domain spectrum masking

Techniques for improving acoustic echo cancellation to attenuate an echo signal generated by a loudspeaker included in a device are described. A system may determine a loudspeaker canceling beam (LCB) (e.g., fixed beam directed to the loudspeaker) and may use the LCB to generate LCB audio data that corresponds to the echo signal. For example, based on a configuration of the loudspeaker relative to microphone(s) of the device, the system may perform simulation(s) to generate a plurality of filter coefficient values corresponding to the loudspeaker. By subtracting the LCB audio data during acoustic echo cancellation, the system may attenuate the echo signal even when there is distortion or nonlinearity or the like caused by the loudspeaker. In some examples, the system may perform acoustic echo cancellation using the LCB audio data and playback audio data.

Acoustic echo cancellation with loudspeaker canceling beamformer

A system configured to improve noise cancellation by reducing attenuation of local speech in proximity to a device. When the local speech is present in both a target signal and a reference signal, performing noise cancellation to remove the reference signal inadvertently attenuates the local speech. To prevent this, the system may perform first noise cancellation to identify frequency bands associated with the local speech and may generate a modified reference signal based on the frequency bands. For example, the system may generate the modified reference signal by applying attenuation to first frequencies associated with the local speech and/or gain to second frequencies that are not associated with the local speech. The system may generate final output audio data by performing noise cancellation using the modified reference signal.

A system configured to improve beamforming by using deep neural networks (DNNs). The system can use one trained DNN to focus on a first person speaking an utterance (e.g., target user) and one or more trained DNNs to focus on noise source(s) (e.g., wireless loudspeaker(s), a second person speaking, other localized sources of noise, or the like). The DNNs may generate time-frequency mask data that indicates individual frequency bands that correspond to the particular source detected by the DNN. Using this mask data, a beamformer can generate beamformed audio data that is specific to a source of noise. The system may perform noise cancellation to isolate first beamformed audio data associated with the target user by removing second beamformed audio data associated with noise source(s).

Multichannel noise cancellation using deep neural network masking

Systems and methods for disabling adaptive echo cancellation functionality for a temporal window are provided herein. In some embodiments, audio data may be received by a voice activated electronic device, where the audio data may include an utterance of a wakeword that may be subsequently followed by additional speech. A start time of when the wakeword began to be uttered may be determined by the voice activated electronic device, and the voice activated electronic device may also send the audio data to a backend system. Adaptive echo cancellation functionality may be disabled at the start time. The backend system may determine an end time of the speech, and may provide an indication to the voice activated electronic device of the end time, which in turn may cause the voice activated electronic device to enable the adaptive echo cancellation functionality at the end time.

Adaptive echo cancellation

An acoustic echo cancellation (AEC) system that detects and compensates for differences in delay times between the AEC system and a set of wireless speakers. The filter coefficients used for AEC are adjusted based on the determined delay time to correct for frequency domain signal rotation.

Multichannel acoustic echo cancellation for wireless applications

An echo cancellation system performs audio beamforming to separate audio input into multiple directions (e.g., target signals) and generates multiple audio outputs using two acoustic echo cancellation (AEC) circuits. A first AEC removes a playback reference signal (generated from a signal sent a loudspeaker) to isolate speech included in the target signals. A second AEC removes an adaptive reference signal (generated from microphone inputs corresponding to audio received from the loudspeaker) to isolate speech included in the target signals. A beam selector receives the multiple audio outputs and selects the first AEC or the second AEC based on a linearity of the system. When linear (e.g., no distortion or variable delay between microphone input and playback signal), the beam selector selects an output from the first AEC based on signal to noise (SNR) ratios. When nonlinear, the beam selector selects an output from the second AEC.

Multichannel acoustic echo cancellation

An acoustic interference cancellation system that performs beamforming using a subset of microphones from a microphone array. For example, a first group of microphones from an array can be used to generate target signals that focus on the direction of the desired speech in the audio and a second group of microphones from the array can be used to generate reference signals that include the environmental noise, audio from a loudspeaker, etc. The reference signals of the second group of microphones can then be used to isolate the actual speech from the target signals of the first group of microphones. The microphone array can be three dimensional, allowing a device to simplify beamforming calculations by selecting subsets of microphones along different planes. In addition, directional microphones and remote microphones may be used to improve a quality of the reference signals.

Growing community of inventors

Morgan Hill, CA, United States of America

Robert Ayrapetian