Tencent America LLC

Chunlei Zhang

Dong Yu

Meng Yu

Yong Xu

Shi-xiong Zhang

Jiachen Lian

Brian Yan

A method, apparatus, and non-transitory computer-readable medium for automatic speech recognition using conditional factorization for bilingual code-switched and monolingual speech may include receiving an audio observation sequence comprising a plurality of frames, the audio observation sequence including audio in a first language or a second language. The approach may further include mapping the audio observation sequence into a first sequence of hidden representations, the mapping being generated by a first encoder corresponding to the first language and mapping the audio observation sequence into a second sequence of hidden representations, the mapping being generated by a second encoder corresponding to the second language. The approach may further include generating a label-to-frame sequence based on the first sequence of hidden representations and the second sequence of hidden representations, using a joint neural network based model.

Conditional factorization for jointly modeling code-switched and monolingual ASR

A method for disentangled variational speech representation learning for voice conversion, performed by at least one processor, is provided. The method includes receiving input speech segments, encoding the input speech segments via a shared encoder to generate a speaker embedding and a content embedding, encoding the posterior distributions of the speaker embedding via a speaker encoder and encoding the posterior distributions of the content embedding via a content encoder to obtain encoded results, and decoding the encoded results by concatenating the encoded results to obtain a reconstructed speech output.

Techniques for disentangled variational speech representation learning for zero-shot voice conversion

A method of acoustic echo suppression using a recurrent neural network, performed by at least one processor, is provided. The method includes receiving a microphone signal and a far-end reference signal, estimating an echo suppressed signal and an echo signal based on the microphone signal and the far-end reference signal, estimating enhancement filters for the microphone signal based on the echo suppressed signal and the echo signal, generating an enhanced signal based on the enhancement filters, and adjusting the enhanced signal using automatic gain control (AGC) and outputting the adjusted signal.

Neuralecho: a self-attentive recurrent neural network for unified acoustic echo suppression, speaker aware speech enhancement and automatic gain control

A method includes receiving, by a first encoder, an original speech segment, receiving, by a second encoder, an augmented speech segment of the original speech segment, generating, by the first encoder, a first speaker representation based on the original speech segment, generating, by the second encoder, a second speaker representation based on the augmented speech segment, and generating a contrastive loss based on the first speaker representation and the second speaker representation.

Progressive contrastive learning framework for self-supervised speaker verification

A method of end-to-end speaker diarization (EESD) using neural speaker clustering, performed by at least one processor, is provided. The method includes generating a set of speech labels corresponding to a set of speakers based on an input stream. The speech labels indicate whether dialogue of a speaker is speech or non-speech. The method further includes generating dialogue based on the set of speakers, extracting speaker embeddings from the dialogue, mapping the speaker embeddings to a cluster identification (ID), an overlapped speech value, or a non-speech value based on a neural network, and outputting EESD labels based on the mapping.

Techniques for end-to-end speaker diarization with generalized neural speaker clustering

An unsupervised text to speech system utilizing a lexicon to map input text to the phoneme sequence, which is expanded to the frame-level forced alignment with a speaker-dependent duration model. An alignment mapping module that converts the forced alignment to the unsupervised alignment (UA). Afterword, a Conditional Disentangled Sequential Variational Auto-encoder (C-DSVAE), serving as the self-supervised TTS AM, takes the predicted UA and a target speaker embedding to generate the mel spectrogram, which is ultimately converted to waveform with a neural vocoder.

Growing community of inventors

Bellevue, WA, United States of America

Chunlei Zhang