Amazon Technologies, Inc.

Andrew Oberlin

Ariya Rastrow

Eli Joshua Fidler

Stanislaw Ignacy Pasko

Kevin Crews

Roland Maximilian Rolf Maas

Michal Kowalczuk

Veerdhawal Pande

Pankaj Sitpure

Prahalad Venkataramanan

Gopinath Chennupati

Anand Mohan

Buddha Puneeth Nandanoor

Santosh Kumar Patro

Milind Mukesh Rao

Gurpreet Singh Chadha

Pawel Zelazko

Cagdas Bak

Paul Anthony Bernhardt

Prasanna H Sridhar

Samarth Batra

Nicholas Matthew Jutila

A first neural network (NN) model may generate labels for training a second NN model. The second NN model may represent instances of a NN model operating on multiple different devices (e.g., decentralized user and/or edge devices). The system may include using a 'teacher' model to process data received by one or more of the devices to generate a labeled dataset. The system may use the labeled dataset and a “student” model to calculate gradient data for updating the student model. The student model may be the same or similar to NN model instances operating on the devices. The system may validate the updated student model to determine, for example, whether it exhibits improved performance when processing the newly received data and/or historical data. The system may distribute the validated update to the devices.

Continuous learning for machine learning models

Some speech processing systems may handle some commands on-device rather than sending the audio data to a second device or system for processing. The first device may have limited speech processing capabilities sufficient for handling common language and/or commands, while the second device (e.g., an edge device and/or a remote system) may call on additional language models, entity libraries, skill components, etc. to perform additional tasks. An intermediate data generator may facilitate dividing speech processing operations between devices by generating a stream of data that includes a first-pass ASR output (e.g., a word or sub-word lattice) and other characteristics of the audio data such as whisper detection, speaker identification, media signatures, etc. The second device can perform the additional processing using the data stream; e.g., without using the audio data. Thus, privacy may be enhanced by processing the audio data locally without sending it to other devices/systems.

Intermediate data for inter-device speech processing

Techniques for an interactive turn-based reading experience are described. A system may take turns reading content, such as a book, with a user. The system may process audio data representing a user reading a portion of the content, determine reading evaluation data, and determine how to proceed for the next turn based on the reading evaluation data. For example, based on the reading evaluation data, the system may read a portion of the content by outputting synthesized speech representing the content, may ask the user re-read a portion of the content, or may ask the user to read a different, smaller portion of the content.

Speech processing techniques

Growing community of inventors

Lynnwood, WA, United States of America

Andrew Oberlin