Amazon Technologies, Inc.

Anshuman Ganguly

Wontak Kim

Srivatsan Kandadai

Trausti Thor Kristjansson

Carlo Murgia

Wai Chung Chu

Spencer Russell

Mrudula V Athi

Alexander M Epstein

A device capable of using image data for purposes of determining a location of a user and audio beam selection to isolate audio in the direction of the user. The beamforming/beam-steering may occur after determining the user's location in order to conserve computing resources that would otherwise have been spent determining beams for non-desired directions. The beamformed audio may be used for speech processing, a communication session involving the device, or other purposes.

Beamforming using image data

A device capable of using data from multiple sensors to determine an estimated position/direction of a user with respect to the device. The device may use estimated position data, along with confidence data, that originated from a plurality of sensors to fuse the data to determine the user's estimated position and comprehensive confidence of the estimated position. The system may use the location information to perform beamforming/beam steering and/or other downstream operations using the comprehensive estimated position.

Beamforming using multiple sensor data

Techniques for improving sound source localization (SSL) are provided. A method for probabilistic SSL using a deep neural network (DNN) may include receiving audio data including a representation of audio such as a wakeword from a microphone array. The audio data may be processed by a DNN to output a plurality of values where each value indicates a probability that the audio originated from a direction corresponding to that value. A sensor may provide computer vision or other data which may be used to inform the plurality of values based on detecting presence of a human or obstacle. A probability that the audio originated from one of the directions of the plurality of directions may be determined based at least in part on the DNN output and the computer vision or other data.

Sound source localization

A device capable of autonomous motion may move in response to a user speaking an utterance, such as a command. Before moving, the device processes audio data received from a microphone array to identify different audio signals arriving at the device from different directions. Based on properties of the audio signals, the device determines which of the audio signals are merely reflections of other audio.

Autonomously motile device with audio reflection detection

A voice-controlled device includes a beamformer for determining audio data corresponding to one or more directions and a beam selector for selecting in which direction a source of target audio lies. The device determines magnitude spectrums for each beam and for each frequency bin in each beam for each frame of audio data. The device determines frame-by-frame changes in the magnitude and filters the changes to smooth them. The device selects the beam having the greatest smoothed change in magnitude as corresponding to speech.

Growing community of inventors

Cambridge, MA, United States of America

Anshuman Ganguly