Amazon Technologies, Inc.

Arent Van Korlaar

Elena Sergeevna Sokolova

Andrew Paul Breen

Nikhil Sharma

Arnaud Vincent Pierre Yves Joly

Antonio Bonafonte

Bartosz Perz

Ioannis Douratsos

Mateusz Aleksander Lajszczak

Panagiotis Agis Oikonomou Filandras

Jonas Felix Ananda Rohnke

Marco Nicolis

Ruizhe Li

Jaime Lorenzo Trueba

Adam Marek Gabrys

Ekaterina Petrova

Jedrzej Sobanski

A target voice dataset may be augmented using speech prediction. Encoder and decoder models may be trained to encode audio data into encoded speech data, and convert it back to audio. The encoded units may include semantic information (e.g., phonemes and/or words) as well as feature data indicating prosody, timbre, speaker identity, speech style, emotion, etc. of speech. An acoustic/semantic language model (ASLM) may be configured to predict encoded speech data in a manner analogous to a language model predicting words; for example, based on preceding encoded speech data. The models may be used to generate synthesized speech samples having voice characteristics (e.g., feature data) similar to those of the target voice dataset. The augmented dataset may be used to train a text-to-speech (TTS) model to reproduce the target voice characteristics, and may improve performance of the TTS model over training with only the original target voice dataset.

Augmenting datasets for training audio generation models

A neural text-to-speech system may be configured to emphasize words. Applying emphasis where appropriate enables the TTS system to better reproduce prosodic characteristics of human speech. Emphasis may make the resulting synthesized speech more understandable and engaging than synthesized speech lacking emphasis. Emphasis may be manually annotated to, and/or predicted from, a source text (e.g., a book). In some implementations, the system may use a generative model such as a variational autoencoder to generate word acoustic embeddings indicating how emphasis is to be reflected in the synthesized speech. A phoneme encoder of the TTS system may process phonemes to generate phoneme embeddings. A decoder may process the word acoustic embeddings and the phoneme embeddings to generate spectrogram data representing the synthesized speech.

Emphasizing portions of synthesized speech

A speech-processing system receives both text data and natural-understanding data (e.g., a domain, intent, and/or entity) related to a command represented in the text data. The system uses the natural-understanding data to vary vocal characteristics in determining spectrogram data corresponding to the text data based on the natural-understanding data.

Synthetic speech processing

Growing community of inventors

London, United Kingdom

Arent Van Korlaar