International Business Machines Corporation

Vyacheslav Shechtman

Alexander Sorin

An example system includes a processor to receive training targets. The training targets include an observed prosody info vector. The processor can train a neural network to predict acoustic sequences based on the training targets. The processor can train a prosody info generator to predict combined prosody info.

Training neural networks to predict acoustic sequences using observed prosody info

Embodiments may provide improved techniques to assess model checkpoint stability on unseen data on-the-fly, so as to prevent unstable checkpoints from being saved, and to avoid or reduce the need for an expensive thorough evaluation. For example, a method may comprise passing a set of input sequences through a checkpoint of a sequence to sequence model in inference mode to obtain a set of generated sequences of feature vectors, determining whether each of a plurality of generated sequences of feature vectors is complete, counting a number of incomplete generated sequences of feature vectors among the plurality of generated sequences of feature vectors, generating a score indicating a stability of the model based on the count of incomplete generated sequences of feature vectors, and storing the model checkpoint when the score indicating the stability of the model is above a predetermined threshold.

Robust checkpoint selection for monotonic autoregressive seq2seq neural generative models

In a trained attentive decoder of a trained Sequence-to-Sequence (seq2seq) Artificial Neural Network (ANN): obtaining an encoded input vector sequence; generating, using a trained primary attention mechanism of the trained attentive decoder, a primary attention vectors sequence; for each primary attention vector of the primary attention vectors sequence: (a) generating a set of attention vector candidates corresponding to the respective primary attention vector, (b) evaluating, for each attention vector candidate of the set of attention vector candidates, a structure fit measure that quantifies a similarity of the respective attention vector candidate to a desired attention vector structure, (c) generating, using a trained soft-selection ANN, a secondary attention vector based on said evaluation and on state variables of the trained attentive decoder; and generating, using the trained attentive decoder, an output sequence based on the encoded input vector sequence and the secondary attention vectors.

Structure-preserving attention mechanism in sequence-to-sequence neural models

An example system includes a processor to receive a linguistic sequence and a prosody info offset. The processor can generate, via a trained prosody info predictor, combined prosody info including a number of observations based on the linguistic sequence. The number of observations include linear combinations of statistical measures evaluating a prosodic component over a predetermined period of time. The processor can generate, via a trained neural network, an acoustic sequence based on the combined prosody info, the prosody info offset, and the linguistic sequence.

Generating acoustic sequences via neural networks using combined prosody info

Embodiments of the present systems and methods may provide techniques that provide the capability to automatically generate allowance intervals for voice personas that meet desired requirements for realism and fidelity. For example, a method for voice persona generation may be implemented in a computer system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, the method comprising: displaying to a user, a plurality of user-selectable voice persona parameters that control features of a synthesized voice signal, and displaying, in conjunction with each of at least some of plurality of user-selectable voice persona parameters, voice transformation allowance intervals of the voice persona parameters, accepting from a user, a selection of at least one user-selectable voice persona parameter, and generating a synthesized voice signal based on the selected at least one user-selectable voice persona parameter.

Growing community of inventors

Haifa, Israel

Vyacheslav Shechtman