Amazon Technologies, Inc.

Rawles LLC

Cisco Technology, Inc.

Kavitha Velusamy

Amit Singh Chhetri

Ramya Gopalan

Philip Ryan Hilmes

Edward Dietz Crump

Wai Chung Chu

Tony David

William Folwell Barton

Michael Alan Pogue

Preethi Narayanan

Kenneth Edgar Hild, Ii

Menashe Haskin

Kurt W Piersol

Gregory Michael Hart

William Spencer Worley, Iii

Ning Yao

Olusanya Temitope Soyannwo

Tina Yung-Ting Chen

Wei Ey Li

Duanpei Wu

Nikko Strom

Qiang Liu

Scott Ian Blanksteen

Vijay Kamarshi

Sowmya Gopalan

Manika Puri

Daniel Christopher Bay

Rahul Agrawal

Robert Warren Sjoberg

Jason Zimmer

Kevin Wayne Arthur

A distributed voice controlled system has a primary assistant and at least one secondary assistant. The primary assistant has a housing to hold one or more microphones, one or more speakers, and various computing components. The secondary assistant is similar in structure, but is void of speakers. The voice controlled assistants perform transactions and other functions primarily based on verbal interactions with a user. The assistants within the system are coordinated and synchronized to perform acoustic echo cancellation, selection of a best audio input from among the assistants, and distributed processing.

Voice controlled system

The systems, devices, and processes described herein may identify a beam of a voice-controlled device that is directed toward a reflective surface, such as a wall. The beams may be created by a beamformer. An acoustic echo canceller (AEC) may create filter coefficients for a reference sound. The filter coefficients may be analyzed to identify beams that include multiple peaks. The multiple peaks may indicate presence of one or more reflective surfaces. Using the amplitude and the time delay between the peaks, the device may determine that it is close to a reflective surface in a direction of the beam.

Beam rejection in multi-beam microphone systems

The systems, devices, and processes described herein may asynchronously transfer audio signals from a voice-controlled device to a remote device. The audio signals may correspond to sound that is captured by multiple microphones of the voice-controlled device, which may then process the audio signals. The audio signals may also be transferred to the remote device for processing. Moreover, a determination of whether the voice-controlled device or the remote device is to process the audio signals may be based at least in part on the bandwidth of a network communicatively coupled to the voice-controlled device. The voice-controlled device may also cache and log the audio signals, and then asynchronously stream the audio signals to the remote device after the audio signals are initially processed, which may be based on the bandwidth of the network. The remote device may utilize the unprocessed audio signals to improve subsequent processing of audio signals.

Asynchronous transfer of audio data

Techniques for tailoring beamforming techniques to environments such that processing resources may be devoted to a portion of an audio signal corresponding to a lobe of a beampattern that is most likely to contain user speech. The techniques take into account both acoustic characteristics of an environment and heuristics regarding lobes that have previously been found to include user speech.

Tailoring beamforming techniques to environments

Techniques for enhancing an acoustic echo canceller based on visual cues are described herein. The techniques include changing adaptation of a filter of the acoustic echo canceller, calibrating the filter, or reducing background noise from an audio signal processed by the acoustic echo canceller. The changing, calibrating, and reducing are responsive to visual cues that describe acoustic characteristics of a location of a device that includes the acoustic echo canceller. Such visual cues may indicate that no human being is present at the location, that some subject(s) are engaged in speaking or sound generating activities, or that motion associated with an echo path change has occurred at the location.

Acoustic echo cancellation using visual cues

An audio device may be configured to produce output audio and to capture input audio for speech recognition. In some cases, a second device may also be used to capture input audio to improve isolation of input audio with respect to the output audio. In addition, acoustic echo cancellation (AEC) may be used to remove components of output audio from input signals of the first and second devices. AEC may be implemented by an adaptive filter based on dynamically optimized filter coefficients. The filter coefficients may be analyzed to detect situations in which the first and second devices are too close to each other, and the user may then be prompted to increase the distance between the two devices.

Detecting device proximities

An augmented reality environment allows interaction between virtual and real objects. Beamforming techniques are applied to signals acquired by an array of microphones to allow for simultaneous spatial tracking and signal acquisition from multiple users. Localization information such as from other sensors in the environment may be used to select a particular set of beamformer coefficients and resulting beampattern focused on a signal source. Alternately, a series of beampatterns may be used iteratively to localize the signal source in a computationally efficient fashion. The beamformer coefficients may be pre-computed.

Signal-enhancing beamforming in an augmented reality environment

A computing system with multiple audio devices local to an environment to provide synchronized audio output to a user. A remote system receives a local audio signal representing sound captured from an environment from a local audio device and a remote audio signal representing sound captured from the environment from the remote audio device. The server determines a delay associated with the local audio signal and a delay associated with the remote audio signals and, based on the delays, synchronizes the audio being output by the local and remote devices.

Synchronizing audio outputs

An audio processing system configured to generate, based at least in part on captured sound, an audio signal that includes a speech component corresponding to a user's speech utterance and an audio component corresponding to audio output of another device is described herein. The audio processing system is also configured to receive a reference signal that corresponds to the audio output of the other device. The reference signal may be received as ultrasonic audio output of the other device or from a remote server. The audio processing device then processes the generated audio signal to remove at least a part of the generated audio signal that corresponds to the reference signal.

Echo cancellation based on shared reference signals

Growing community of inventors

San Jose, CA, United States of America

Kavitha Velusamy