International Business Machines Corporation

Nuance Communications, Inc.

Motorola Corporation

Motorola Mobility LLC

Google Technology Holdings LLC

Alexander Sorin

Slava Shechtman

Tenkasi Vaideeswaran Ramabadran

Vyacheslav Shechtman

Baiju Dhirajlal Mandalia

Ron Hoory

Alexander Faisman

Andrew M Rosenberg

Shay Ben-David

Zvi Kons

Zohar Sivan

Dan Chazan

Vincent Pollet

Raul Fernandez

Eugene Wallach

Andrei Heilper

Yaniv Gal

In a trained attentive decoder of a trained Sequence-to-Sequence (seq2seq) Artificial Neural Network (ANN): obtaining an encoded input vector sequence; generating, using a trained primary attention mechanism of the trained attentive decoder, a primary attention vectors sequence; for each primary attention vector of the primary attention vectors sequence: (a) generating a set of attention vector candidates corresponding to the respective primary attention vector, (b) evaluating, for each attention vector candidate of the set of attention vector candidates, a structure fit measure that quantifies a similarity of the respective attention vector candidate to a desired attention vector structure, (c) generating, using a trained soft-selection ANN, a secondary attention vector based on said evaluation and on state variables of the trained attentive decoder; and generating, using the trained attentive decoder, an output sequence based on the encoded input vector sequence and the secondary attention vectors.

Structure-preserving attention mechanism in sequence-to-sequence neural models

Embodiments of the present systems and methods may provide techniques that provide the capability to automatically generate allowance intervals for voice personas that meet desired requirements for realism and fidelity. For example, a method for voice persona generation may be implemented in a computer system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, the method comprising: displaying to a user, a plurality of user-selectable voice persona parameters that control features of a synthesized voice signal, and displaying, in conjunction with each of at least some of plurality of user-selectable voice persona parameters, voice transformation allowance intervals of the voice persona parameters, accepting from a user, a selection of at least one user-selectable voice persona parameter, and generating a synthesized voice signal based on the selected at least one user-selectable voice persona parameter.

Voice transformation allowance determination and representation

A computer system receives a training set of voice data captured from one or more speakers and classified with one or more categorical prosodic labels according to one or more prosodic categories. The computer system transforms the voice data of the training set while preserving one or more portions of the voice data that determine the one or more categorical prosodic labels to automatically form a new training set of voice data automatically classified with the one or more categorical prosodic labels according to the one or more prosodic categories. The computer system augments a database comprising the training set with the new training set for training a speech model of an artificial intelligence system.

Voice-transformation based data augmentation for prosodic classification

A method comprising: receiving an utterance, an original pitch contour of the utterance, and a target pitch contour for the utterance, wherein the utterance comprises a plurality of consecutive frames, and wherein at least one of said frames is a voiced frame; calculating an original intensity contour of said utterance; generating a pitch modified utterance based on the target pitch contour; calculating an intensity modification factor for each of said frames, based on said original pitch contour and on said target pitch contour, to produce a sequence of intensity modification factors corresponding to said plurality of consecutive frames; calculating a final intensity contour for said utterance by applying said intensity modification factors to said original intensity contour; and generating a coherently modified speech signal by time dependent scaling of the intensity of said pitch modified utterance according to said final intensity contour.

Coherent pitch and intensity modification of speech signals

Coherently-modified speech signal generation by time-dependent scaling of intensity of a pitch-modified utterance

Some embodiments relate to techniques for encoding an audio signal represented by a plurality of frames including a first frame. The techniques include using at least one computer hardware processor to perform: obtaining an initial discrete spectral representation of the first frame; obtaining a primary discrete spectral representation of the initial discrete spectral representation at least in part by estimating a phase envelope of the initial discrete spectral representation and evaluating the estimated phase envelope at a discrete set of frequencies; calculating a residual discrete spectral representation of the initial discrete spectral representation based on the initial discrete spectral representation and the primary discrete spectral representation; and encoding the residual discrete spectral representation using a plurality of codewords.

Systems and methods for encoding audio signals

An embodiment according to the invention provides a capability of automatically predicting how favorable a given speech signal is for statistical modeling, which is advantageous in a variety of different contexts. In Multi-Form Segment (MFS) synthesis, for example, an embodiment according to the invention uses prediction capability to provide an automatic acoustic driven template versus model decision maker with an output quality that is high, stable and depends gradually on the system footprint. In speaker selection for a statistical Text-to-Speech synthesis (TTS) system build, as another example context, an embodiment according to the invention enables a fast selection of the most appropriate speaker among several available ones for the full voice dataset recording and preparation, based on a small amount of recorded speech material.

System and method for automatic prediction of speech suitability for statistical modeling

A method, system and computer program product are provided for enhancement of speech synthesized by a statistical text-to-speech (TTS) system employing a parametric representation of speech in a space of acoustic feature vectors. The method includes: defining a parametric family of corrective transformations operating in the space of the acoustic feature vectors and dependent on a set of enhancing parameters; and defining a distortion indictor of a feature vector or a plurality of feature vectors. The method further includes: receiving a feature vector output by the system; and generating an instance of the corrective transformation by: calculating a reference value of the distortion indicator attributed to a statistical model of the phonetic unit emitting the feature vector; calculating an actual value of the distortion indicator attributed to feature vectors emitted by the statistical model of the phonetic unit emitting the feature vector; calculating the enhancing parameter values depending on the reference value of the distortion indicator, the actual value of the distortion indicator and the parametric corrective transformation; and deriving an instance of the corrective transformation corresponding to the enhancing parameter values from the parametric family of the corrective transformations. The instance of the corrective transformation may be applied to the feature vector to provide an enhanced feature vector.

Statistical enhancement of speech output from a statistical text-to-speech synthesis system

A method for estimating high-order Mel Frequency Cepstral Coefficients, the method comprising initializing any of N&#x2212;L high-order coefficients (HOC) of an MFCC vector of length N having L low-order coefficients (LOC) to a predetermined value, thereby forming a candidate MFCC vector, synthesizing a speech signal frame from the candidate MFCC vector and a pitch value, and computing an N-dimensional MFCC vector from the synthesized frame, thereby producing an output MFCC vector.

Restoration of high-order Mel frequency cepstral coefficients

A method for processing a speech signal includes dividing the speech signal into a succession of frames, identifying one or more of the frames as click frames, and extracting phase information from the click frames. The speech signal is encoded using the phase information. Methods are also provided for modeling phase spectra of voiced frames and click frames.

Speech synthesis using complex spectral modeling

An ASR-aided transcription system with segmented feedback training is provided, the system including a transcription process manager configured to extract a first segment and a second segment from an audio input of speech uttered by a speaker, and an ASR engine configured to operate in a first speech recognition mode to convert the first speech segment into a first text transcript using a speaker-independent acoustic model and a speaker-independent language model, operate in a first training mode to create a speaker-specific acoustic model and a speaker-specific language model by adapting the speaker-independent acoustic model and the speaker-independent language model using either of the first segment and a corrected version of the first text transcript, and operate in a second speech recognition mode to convert the second speech segment into a second text transcript using the speaker-specific acoustic model and the speaker-specific language model.

ASR-aided transcription with segmented feedback training

Methods and systems are provided for remote tuning and debugging of an automatic speech recognition system. Trace files are generated on-site from input speech by efficient, lossless compression of MFCC data, which is merged with compressed pitch and voicing information and stored as trace files. The trace files are transferred to a remote site where human-intelligible speech is reconstructed and analyzed. Based on the analysis, parameters of the automatic speech recognition system are remotely adjusted.

Remote tracing and debugging of automatic speech recognition servers by speech reconstruction from cepstra and pitch information

Restoration of high-order Mel Frequency Cepstral Coefficients

Estimating a speech signal pitch frequency by determining a speech signal frame line spectrum including spectral lines having respective line amplitudes and frequencies, selecting a predefined number of spectral lines having highest amplitudes, fewer then the total number of the spectral lines, calculating a preliminary utility function over a pitch frequency range to provide a preliminary utility function value for each pitch frequency in the range measuring the compatibility of the selected spectral lines with the pitch frequency, identifying a predefined number of preliminary pitch frequency candidates at least partly responsive to the preliminary utility function, where each candidate is a local maximum of the preliminary utility function, calculating a final utility score for each of the candidates, and selecting any of the candidates to be an estimated pitch frequency of the speech signal at least partly responsive to any of the final utility scores.

Growing community of inventors

Haifa, Israel

Alexander Sorin