Systems and methods for generating synthesized speech responses to voice inputs by training a neural network model based on the voice input prosodic metrics and training voice inputs

Abstract

The system trains a model to provide information used to provide a synthesized speech response to a voice input. The model takes as input prosodic information that may include pitch, note, duration, prominence, timbre, rate, and rhythm, for example. The system receives a plurality of voice inputs, each associated with prosodic metric, as well as a plurality of responses, each also associated with prosodic metrics. The system trains the model based on the plurality of voice inputs, the plurality of responses, the prosodic metrics of the voice inputs, and the prosodic metrics of the responses such that the model outputs information used to generate the response. The model may also take as input user profile information, emotion metrics, and transition information to generate output. The output of the training model may be used by the system to provide synthesized speech responses having relevant prosodic character to received voice inputs.