Amazon Technologies, Inc.

Nilaksh Das

Sravan Babu Bodapati

Katrin Kirchhoff

Monica Lakshmi Sunkara

Srikanth Ronanki

Jeffrey John Farris

Nicholas G Aldridge

Devang Kulshreshtha

Jinglun Cai

Systems and methods are described for converting speech to text. In one aspect, a method for converting speech to text includes generating an initial transcript of an audio or video file. At least one named entity may be extracted from the initial transcript using entity recognition. A subset of nodes of a knowledge graph that include the at least one named entity may be selected, where the subset of nodes of the knowledge graph correspond to a set of named entities. The method may further include encoding the set of named entities to generate a set of entity embeddings. The speech in the audio or video file may then be decoded using the set of entity embeddings to produce a final transcript of the audio file.

Infusing knowledge graphs into automatic speech recognition

Bias may be removed from automatic speech recognition model predictions using internal language model estimates. Audio data may be received for speech recognition. The audio data may be processed both through an automatic speech recognition model to produce original word token predictions and masked in different portions of the audio data to produce other word token predictions for the masked audio. A comparison of the original word token predictions and the other word token predictions may provide an estimate of an internal language model for the automatic speech recognition model. This estimate can be used to modify the original word token predictions to remove the lexical bias and produce a speech prediction.

Removing bias from automatic speech recognition models using internal language model estimates

Growing community of inventors

Seattle, WA, United States of America

Nilaksh Das