Amazon Technologies, Inc.

Venkata Snehith Cherukuri

Joseph White

Vinodth Kumar Mohanam

Deepak Uttam Shah

Ravi Kiran Rachakonda

Rami Faris Habal

Maziyar Khorasani

Menghan Li

Lalithkumar Rajendran

Joseph Pedro Tavares

Charles James Torbert

Gang Lan

Mckay Clawson

Vijay Shankar Tennety

Karim Helwani

This disclosure describes, in part, techniques for determining device groupings, or clusters, for multiple voice-enabled devices. The device clusters may be determined based on metadata data for audio signals (or audio data) generated by each of the multiple voice-enabled devices. For example, a remote system may analyze timestamp data for the audio signals received from the devices, and determine that the devices detected the same voice command of a user based on the timestamp data indicating that the audio signals were received within a threshold period of time from each other. Additionally, the remote system may analyze other metadata of the audio data, such as signal-to-noise (SNR) values, and determine that the SNR values are within a threshold value. The remote system may determine device clusters for the voice-enabled devices of a user based on these, and potentially other, types of metadata of the audio signals.

Determining device groups

This disclosure describes, in part, context-based device arbitration techniques to select a voice-enabled device from multiple voice-enabled devices to provide a response to a command included in a speech utterance of a user. In some examples, the context-driven arbitration techniques may include determining a ranked list of voice-enabled devices that are ranked based on audio signal metric values for audio signals generated by each voice-enabled device, and iteratively moving through the list to determine, based on device states of the voice-enabled devices, whether one of the voice-enabled devices can perform an action responsive to the command. If the voice-enabled devices that detected the speech utterance are unable to perform the action responsive to the command, all other voice-enabled devices associated with an account may be analyzed to determine whether one of the other voice-enabled devices can perform the action responsive to the command in the speech utterance.

Context-based device arbitration

This disclosure describes techniques for identifying a voice-enabled device from a group of voice-enabled devices to respond to a speech utterance of a user. A speech-processing system may receive an audio signal representing the speech utterance captured in an environment of a voice-enabled device, and identify another voice-enabled device located in the environment. The system may analyze the audio signal using a different natural-language-understanding model for each of the voice-enabled devices to identify an intent for each of the voice-enabled devices to respond to the speech utterance. The system may determine confidence scores that the intents are responsive to the speech utterance, and select the intent with the highest confidence score. The system may use the selected intent to generate a command for the corresponding voice-enabled device to respond to the user.

Device selection from audio data

This disclosure describes techniques for selecting which device to respond to user speech within an environment that includes multiple devices. To enable this selection, the techniques described herein create a model indicating a topology of the devices within the environment. To do so, features associated with the user speech are generated and used to generate multi-dimensional points, each representing a corresponding user utterance. The techniques may then group these multi-dimensional points into clusters before projecting the resulting clusters into two dimensions. The two-dimensional clusters may then be used to generate a model (e.g., a Voronoi diagram) representing regions of the environment, with the centroid of each region being associated with a respective device. When a user makes a subsequent request, the user may be localized within one of the regions and the device associated with that region may be chosen to respond to the user.

Determining devices to respond to user requests

Growing community of inventors

San Jose, CA, United States of America

Venkata Snehith Cherukuri