Amazon Technologies, Inc.

Srivatsan Kandadai

Trausti Thor Kristjansson

Amit Singh Chhetri

Mark A Lawrence

Anna Chen Santos

Miroslav Ristic

Joseph Pedro Tavares

Valere Joseph Vanderschaegen

Balsa Laban

Wontak Kim

Anshuman Ganguly

Donald L Cantrell, Jr

Mohamed Farouk Mansour

Tarun Pruthi

Minje Kim

Navin Chatlani

Harsha Inna Kedage Rao

Byoungsam Woo

Mark Lawrence

Md Tamzeed Islam

Stefan Vlaski

Ananth Raghavendra

Donald L Cantrell

A device capable of using image data for purposes of determining a location of a user and audio beam selection to isolate audio in the direction of the user. The beamforming/beam-steering may occur after determining the user's location in order to conserve computing resources that would otherwise have been spent determining beams for non-desired directions. The beamformed audio may be used for speech processing, a communication session involving the device, or other purposes.

Beamforming using image data

Techniques for presence detection based on multi modal sensors are described. In an example, a computer system determines first data indicating a first prediction of presence detection within a space. The first data is generated based on first sensor data. The computer system determines second data indicating a second prediction of presence detection within the space. The second data is generated based on second sensor data. The second sensor data is of a different type than the first sensor data. The computer system generates third data indicating a third prediction of presence detection within the space based on the first data, the second data, and a fusion model. The fusion model is configured to generate the third data by at least using associations between presence detections, latent variables, and observed variables. The observed variables correspond to the first data and the second data. The latent variables include sensor-triggering events.

Presence detection based on multiple sensors

A device capable of using data from multiple sensors to determine an estimated position/direction of a user with respect to the device. The device may use estimated position data, along with confidence data, that originated from a plurality of sensors to fuse the data to determine the user's estimated position and comprehensive confidence of the estimated position. The system may use the location information to perform beamforming/beam steering and/or other downstream operations using the comprehensive estimated position.

Beamforming using multiple sensor data

An autonomous mobile device (AMD) moves through a physical space without human intervention. Data from sensors on the AMD is used to determine if the AMD has collided with an obstacle. In one implementation the sensors may include a microphone. A collision may produce in the structure of the AMD a characteristic sound that is detected by the microphone and recognized as indicative of a collision. In another implementation information about velocity of the AMD and output from an inertial measurement unit (IMU) may be used to determine a characteristic change in motion that is indicative of a collision. Data from the microphone and the IMU may be combined to improve the reliability of collision detection.

System for collision detection in autonomous mobile device

A system configured to perform deep adaptive acoustic echo cancellation (AEC) to improve audio processing. Due to mechanical noise and continuous echo path changes caused by movement of a device, echo signals are nonlinear and time-varying and not fully canceled by linear AEC processing alone. To improve echo cancellation, deep adaptive AEC processing integrates a deep neural network (DNN) and linear adaptive filtering to perform echo and/or noise removal. The DNN is configured to generate a nonlinear reference signal and step-size data, which the linear adaptive filtering uses to generate output audio data representing local speech. The DNN may generate the nonlinear reference signal by generating mask data that is applied to a microphone signal, such that the reference signal corresponds to a portion of the microphone signal that does not include near-end speech.

Deep adaptive acoustic echo cancellation

A computer-implemented method includes receiving, at a microphone of a voice-controlled device, a speech input, generating an electrical signal having a first gain level that is below a gain threshold for audible detection by a user, transmitting the electrical signal to the speaker and detecting, by the microphone, an audio signal that includes a combination of ambient noise and a probe audio signal, wherein the probe audio signal is output by the speaker based on the electrical signal. The method further includes determining a power level of the probe audio signal and determining a state of the display based on the power level of the probe audio signal.

Detection of TV state using sub-audible signal

A device capable of autonomous motion includes a residual echo suppressor for suppressing echoes caused by an output reference signal. When the device outputs audio while moving with a velocity, it may receive echoes that are Doppler-shifted due to the motion. The residual echo suppressor generates estimated residual error data based on phase-shifted reference data to account for and suppress the Doppler-shifted echoes.

Autonomously motile device with residual echo suppression

A computer-implemented method includes receiving, at a microphone of a voice-controlled device, a speech input from a user and determining, by the voice-controlled device, that a power state of an AV display device that is coupled to the voice-controlled device is an ON or OFF state. Based on the user intent determined from the speech input and the power state of the AV display, the voice-controlled device sends data to the AV display device to switch the AV display device ON or OFF. The method can further include receiving, by the voice-controlled device, content from a content source location and sending the content to the AV display device via an AV port.

Voice-controlled multimedia device

A mobile device capable of capturing voice commands includes a beamformer for determining audio data corresponding to one or more directions and a beam selector for selecting in which direction a source of target audio lies. The device determines, based on data from one or more sensors, an angle through which the device has rotated. Based on the angle and one or more rotation-compensation functions, the device interpolates audio data corresponding to the one or more directions to compensate for the rotation such that the direction corresponding to the source of target audio remains selected.

Beamformer rotation

A voice-controlled device includes a beamformer for determining audio data corresponding to one or more directions and a beam selector for selecting in which direction a source of target audio lies. The device determines magnitude spectrums for each beam and for each frequency bin in each beam for each frame of audio data. The device determines frame-by-frame changes in the magnitude and filters the changes to smooth them. The device selects the beam having the greatest smoothed change in magnitude as corresponding to speech.

Audio beam selection

A device capable of moving a component of the device motion is capable of determining audio data corresponding to a direction relative to the device ('beamforming'). When the component moves, the device determines a new position of the component and selects one of a set of beamforming filter coefficients corresponding to the position. Using the filter coefficients, the device determines the directional audio data.

Autonomously motile device with beamforming

A system that performs wall detection, range estimation, corner detection and/or angular estimation. The system may determine an aggregate impulse response (e.g., impulse response of all components in a room) and may perform a deconvolution to remove a system impulse response (e.g., impulse response associated with loudspeaker(s) and microphone(s)). Thus, the system may use a sparse deconvolution algorithm to estimate a room impulse response (e.g., determine acoustic characteristics of the room). The system may detect a peak in the room impulse response and determine a distance and/or direction to an acoustically reflective surface based on the peak.

Growing community of inventors

San Jose, CA, United States of America

Srivatsan Kandadai