Amazon Technologies, Inc.

Sravan Babu Bodapati

Katrin Kirchhoff

Monica Lakshmi Sunkara

Yahor Pushkin

Rishita Rajal Anubhai

Sunil Mallya Kasaragod

Ashish Vishwanath Shenoy

John Baker

Yaser Al-Onaizan

Varun Sembium Varadarajan

Anubhav Mishra

Pu Paul Zhao

Srikanth Ronanki

Mohit Gupta

Harshal Pimpalkhute

Vivek Govindan

Deepthi Devaiah Devanira

Jeffrey John Farris

Nilaksh Das

Rahul Gupta

Janna Hamaker

Ankur Gandhe

Ganesh Kumar Gella

Sahika Genc

Dimitrios Soulios

David Anthony Leen

Asem Rustum

Tao Sun

Yue Yang

Georgiana Dinu

Syed Kashif Hussain Shah

Kalpit Dixit

Abhinav Goyal

Sameer Karnik

Julian E S Salazar

Ravindra Manjunatha

Ravi Teja Gadde

Bangrui Liu

Sriraghavendra Ramaswamy

Gururaj Narayanan

Jinglun Cai

Nicholas G Aldridge

Yibin Wang

K P N V D S Siva Rama

Calvin Xuan Li

Dhanush Bekal Kannangola

Venkatraman Kalyanapasupathy

John Hambacher

Alok Kumar Parmesh

Saket Dingliwal

Devang Kulshreshtha

Christian Egon Berkhoff Dossow

Chaitanya Shivade

Himalay Mohanlal Joriwal

Silviu Mihai Burz

Sai Madhuri Bhavirisetty

Yide Zou

Abhinav Kumar

Christopher Michael Hittner

Liangwei Guo

Yilai Su

Orestis Lykouropoulos

Anuj Gautam Surana

Brandon Louis Mendez

Akshay Nalwaya

Chengshun Zhang

Language models may be dynamically updated for trending entities based on tuning data for particular users. A user may provide specific tuning data associated with trending entities within a class to generate a weight map for a language model. A class based model may be trained using the weight map specific for the user for the trending entities. Additionally, weights may be further boosted using a boosting language model to emphasize the trending entities.

Dynamic language model updates with boosting

Systems and methods are described for converting speech to text. In one aspect, a method for converting speech to text includes generating an initial transcript of an audio or video file. At least one named entity may be extracted from the initial transcript using entity recognition. A subset of nodes of a knowledge graph that include the at least one named entity may be selected, where the subset of nodes of the knowledge graph correspond to a set of named entities. The method may further include encoding the set of named entities to generate a set of entity embeddings. The speech in the audio or video file may then be decoded using the set of entity embeddings to produce a final transcript of the audio file.

Infusing knowledge graphs into automatic speech recognition

Bias may be removed from automatic speech recognition model predictions using internal language model estimates. Audio data may be received for speech recognition. The audio data may be processed both through an automatic speech recognition model to produce original word token predictions and masked in different portions of the audio data to produce other word token predictions for the masked audio. A comparison of the original word token predictions and the other word token predictions may provide an estimate of an internal language model for the automatic speech recognition model. This estimate can be used to modify the original word token predictions to remove the lexical bias and produce a speech prediction.

Removing bias from automatic speech recognition models using internal language model estimates

Techniques for utilizing a federated learning service are described. An exemplary method includes causing a development of a deployable machine learning model using at least two devices, the development of the deployable machine learning model including: providing an initial machine learning model or algorithm to the at least two devices external to the provider network, causing each of the at least two devices external to the provider network to locally train the initial machine learning model or algorithm using training data to each generate a modified version of the initial machine learning model, determining updates between the initial model and the generated modified versions of the initial machine learning model, and applying the determined updates to the initial model to generate the candidate machine learning model.

Federated learning service in a provider network and training machine learning models using devices external to the provider network

Techniques for at least the generation of a chatbot built from a custom vocabulary and to use runtime hints during inference are described. In some examples, the generation of the chatbot includes receiving a request to build a chatbot using a bot definition and a custom vocabulary, wherein the chatbot is to use runtime hints during usage; building the chatbot from the bot definition and custom vocabulary by at least: generating automatic speech recognition (ASR) artifacts to be used in decoding audio input into the chatbot into text for at least one other component of the chatbot to use in determining a next act to be performed, the ASR artifacts including artifacts that use the custom vocabulary and artifacts that do not use the custom vocabulary, and storing the ASR artifacts.

Dynamically selectable automated speech recognition using a custom vocabulary

Techniques for managing access to sensitive data in transcriptions are described. A method for managing access to sensitive data in transcriptions may include receiving a request to generate a redacted transcript of content, obtaining a transcript of the content, sending at least a portion of the transcript to a model endpoint to identify sensitive entities in the transcript, receiving an inference response identifying one or more sensitive entities in the transcript, and generating the redacted transcript based at least one the transcript and the inference response.

Managing access to sensitive data in transcriptions

A set of alternative vocal input styles for specifying a parameter of a dialog-driven application is determined. During execution of the application, an audio prompt requesting input in one of the styles is presented. A value of the parameter is determined by applying a collection of analysis tools to vocal input obtained after the prompt is presented. A task of the application is initiated using the value.

Dialog-driven applications supporting alternative vocal input styles

Techniques for personalized batch and streaming speech-to-text transcription of audio reduce the error rate of automatic speech recognition (ASR) systems in transcribing rare and out-of-vocabulary words. The techniques achieve personalization of connectionist temporal classification (CT) models by using adaptive boosting to perform biasing at the level of sub-words. In addition to boosting, the techniques encompass a phone alignment network to bias sub-word predictions towards rare long-tail words and out-of-vocabulary words. A technical benefit of the techniques is that the accuracy of speech-to-text transcription of rare and out-of-vocabulary words in a custom vocabulary by automatic speech recognition (ASR) system can be improved without having to train the ASR system on the custom vocabulary. Instead, the techniques allow the same ASR system trained on a base vocabulary to realize the accuracy improvements for different custom vocabularies spanning different domains.

Personalized batch and streaming speech-to-text transcription of audio

Portions of text data generated from inverse text normalization may be redacted. Text data for redaction may be obtained. One or more inverse text normalization models may be applied to the text data to generate normalized text data. A machine learning model, trained to recognize text for redaction, may be applied to identify portions of the normalized text data for redaction. The identified portions may be redacted and the redacted normalized text provided to a destination.

Redacting portions of text transcriptions generated from inverse text normalization

Domain-specific parameters may be used for tuning speech processing. A pre-trained transformer-based language model may train domain-specific parameters using domain-specific unlabeled text data. This domain-specific parameters can then be appended to candidate texts produced by a speech model on received speech data and input to the transformer-based language model to score the candidate texts. The scores of the candidate texts determined using the pre-trained transformer-based language model can then be used to select a candidate text for further speech processing.

Domain-specific parameter pre-fixes for tuning automatic speech recognition

A data source configured to provide a representation of an environment of one or more agents is identified. Using a data set obtained from the data source, a neural network-based reinforcement learning model with one or more attention layers is trained. Importance indicators generated by the attention layers are used to identify actions to be initiated by an agent. A trained version of the model is stored.

Attention-based deep reinforcement learning for autonomous agents

Various embodiments of a hierarchical system or method of identifying sensitive content in data is described. In some embodiments, sensitive data classifiers local to a data storage system can analyze a plurality of data items and classify at least some data items as potentially containing sensitive data. The sensitive data classifiers can provide the classified data items to a separate sensitive data discovery component. The sensitive data discovery component can, in some embodiments, obtain the classified data items, perform a sensitive data location analysis on the classified data items to identify a location of sensitive data within some of the classified data items, and generate location information for the sensitive data within the data items containing sensitive data. The sensitive data discovery component can provide to a destination this information, in some embodiments, where the destination might redact, tokenize, highlight, or perform other actions on the located sensitive data.

Hierarchical system and method for identifying sensitive content in data

Techniques for multi-label document classification are described. Clustering is used to cluster labels in a set. A machine learning model including a multi-label classifier for each cluster is created, the multi-label classifier for a given cluster to classify a document with one or more of the labels in the cluster.

Multi-label document classification for documents from disjoint class sets

Techniques for creating a text classifier machine learning (ML) model are described. According to some embodiments, a language processing service finetunes a language ML model on unlabeled documents of a user, and then trains that finetuned language ML model on labeled documents of the user to be a text classifier that is customized for that user's domain, e.g., the user's documents. Additionally, the finetuned language ML model may be trained on labeled documents of the user, for prediction objectives for unlabeled data, before being trained as the text classifier.

Creating text classification machine learning models

Growing community of inventors

Bellevue, WA, United States of America

Sravan Babu Bodapati