Amazon Technologies, Inc.

Borham Lee

Wai Chung Chu

Carlo Murgia

Mohamed Farouk Mansour

Shobha Devi Kuruba Buchannagari

Carlos Renato Nakagawa

A system configured to improve SSL processing and/or target goal detection by fusing SSL data with object information to generate a combined target likelihood estimate that takes into account what the device knows about the surrounding environment. For example, the device may generate object information by performing object detection, floorplan estimation, distance measurements, and/or the like. Using this object information, the device may calculate a likelihood estimate value for each direction around the device, with known objects (e.g., walls) corresponding to low likelihood values. In response to an acoustic event (e.g., wakeword detection), the device may fuse the target likelihood estimates generated using SSL data and/or object information to generate the combined target likelihood estimate. Thus, the combined target likelihood estimate enables the device to accurately associate the acoustic event with a corresponding SSL track (e.g., direct sound) and ignore reflections caused by objects in the environment.

Target likelihood fusion

A system configured to perform audio type detection for sound source localization (SSL) data is provided. A device processes audio data representing sounds from multiple sound sources to determine SSL data that distinguishes between each of the sound sources. To identify an audio type associated with a sound source and/or track individual sound sources over time, the device can determine a correlation between the SSL data and an audio event. Examples of audio events include a wakeword component detecting a wakeword or an acoustic event detector detecting a particular acoustic event. The device may determine a correlation between wakeword data indicating that a wakeword is represented in the audio data and SSL data for each individual sound source. The device may then identify a sound source that is most strongly correlated to the wakeword data and associate the wakeword and a corresponding voice command with the sound source.

Sound source localization audio type detection

A system configured to improve track selection while performing audio type detection using sound source localization (SSL) data is provided. A device processes audio data representing sounds from multiple sound sources to determine SSL data that distinguishes between each of the sound sources. The system detects an acoustic event and performs SSL track selection to select the sound source that corresponds to the acoustic event based on input features. To improve SSL track selection, the system detects current conditions of the environment and determines adaptive weight values that vary based on the current conditions, such as a noise level of the environment, whether playback is detected, whether the device is located near one or more walls, etc. By adjusting the adaptive weight values, the system improves an accuracy of the SSL track selection by prioritizing the input features that are most predictive during the current conditions.

Sound source localization

A device has a microphone array that acquires sound data and a camera that acquires image data. A portion of the device may be moveable by one or more actuators. Responsive to the user, the portion of the device is moved toward an estimated direction of the user. The estimated direction is based on sensor data including the sound data and the image data. First variance values for individual sound direction values are calculated. Data derived from the image data or data from other sensors may be used to modify the first variance values and determine second data comprising second variances. The second data may be processed to determine the estimated direction of the user. For example, the second data may be processed by both a forward and a backward Kalman filter, and the output combined to determine an estimated direction toward the user.

Growing community of inventors

Sunnyvale, CA, United States of America

Borham Lee