Amazon Technologies, Inc.

Sundararajan Srinivasan

Spyridon Matsoukas

Arindam Mandal

Aparna Khare

Rohit Prasad

Krishna Subramanian

Gregory Michael Hart

Hongda Mao

Vishwanathan Krishnamoorthy

Sai Sailesh Kopuri

Chu-Cheng Hsieh

George Yu-Chien Lin

John Moore

Techniques for enrolling a user in a system's user recognition functionality without requiring the user speak particular speech are described. The system may determine characteristics unique to a user input. The system may generate an implicit voice profile from user inputs having similar characteristics. After an implicit voice profile is generated, the system may receive a user input having speech characteristics similar to that of the implicit voice profile. The system may ask the user if the user wants the system to associate the implicit voice profile with a particular user identifier. If the user responds affirmatively, the system may request an identifier of a user profile (e.g., a user name). In response to receiving the user's name, the system may identify a user profile associated with the name and associate the implicit voice profile with the user profile, thereby converting the implicit voice profile into an explicit voice profile.

Speech based user recognition

A first neural network model of a user device processes audio data to extract audio embeddings that represent vocal characteristics of a user of an utterance represented in the audio data. The audio embeddings may then be hashed to remove characteristics specific to the user while still maintaining a unique set of characteristics. The hashed embeddings may be sent to a remote system, which may use them to identify the user.

Speech processing using embedding data

Techniques for labeling user inputs for updating user recognition voice profiles are described. A system may leverage various signals, generated during or after processing of a user input, to retroactively determine which user spoke the user input. For example, after the system receives the user input, the user may provide the system with non-spoken user verification information. Based on such user verification information, the system may label the previously spoken user input as originating from the particular user. The system may also or alternatively use system usage history to retroactively label user inputs.

Voice profile updating

Techniques for updating voice profiles used to perform user recognition are described. A system may use clustering techniques to update voice profiles. When the system receives audio data representing a spoken user input, the system may store the audio data. Periodically, the system may recall, from storage, audio data (representing previous user inputs). The system may identify clusters of the audio data, with each cluster including similar or identical speech characteristics. The system may determine a cluster is substantially similar to an existing voice profile. If this occurs, the system may create an updated voice profile using the original voice profile and the cluster of audio data.

Systems, methods, and devices related to establishing voice identity profiles for use with voice-controlled devices are provided. The embodiments disclosed enhance user experience by customizing the enrollment process to utilize voice recognition for each user based on historical information which can be used in the selection process of phrases a user speaks during enrollment of a voice recognition function or skill. The selection process can utilize phrases that have already been spoken to the electronic device; it can utilize phrases, contacts, or other personalized information it can obtain from the user account of the person enrolling; it can use any of the information just described to select specific words to enhance the probably of achieving higher phonetic matches based on words the individual user is more likely to speak to the device.

Growing community of inventors

Sunnyvale, CA, United States of America

Sundararajan Srinivasan