Microsoft Technology Licensing, LLC

Tencent America LLC

Tencent Technology (shenzhen) Company Limited

Microsoft Corporation

Micriosoft Corporation

Dong Yu

Li Deng

Alejandro Acero

Chao Weng

Chengzhu Yu

Meng Yu

Heng Lu

Yifan Gong

Dan Su

Jia Cui

Jun Wang

Yun-Cheng Ju

Jinyu Li

James G Droppo

Ye-Yi Wang

Gang Li

Yong Xu

Chunlei Zhang

Michael Lewis Seltzer

Frank Torsten Bernd Seide

Guangsen Wang

Yanmin Qian

Shi-Xiong Zhang

Peidong Wang

Milind V Mahajan

Xiaodong He

Geoffrey G Zweig

Mei-Yuh Hwang

Xiaolong Li

Kaisheng N Yao

Jianshu Chen

Peter K L Mau

Dian Yu

Jie Chen

Jian Wu

Frank Seide

Kun Xu

Sangwoo Cho

Shi-xiong Zhang

Jiachen Lian

Xiaoyang Wang

Peter Mau

Yan Zhang

Xuedong D Huang

Ruhi Sarikaya

Kuansan Wang

Li Qin Shen

Ying Shan

Julian James Odell

Xiao Li

Yonggen Ling

Asela J Gunawardana

Shan Yang

Yu Shrike Zhang

Adam Clyde Eversole

Xiang Li

Haozhi Huang

John C Langford

Jian Jiao

Shixiong Zhang

Jian Xue

Shiyin Kang

Anoop Deoras

Lianwu Chen

Fei Wang

Xie Chen

Alex Acero

Jui-Ting Huang

Hang Su

Abdel-rahman Samir Abdel-rahman Mohamed

Shizhen Wang

Xiaoman Pan

Gregoire Mesnil

Jiawei Li

Hongming Zhang

Wenlin Yao

Kaiqiang Song

Hao Zhang

Lifeng Jin

Thomas Ryan Hoens

Haijing Wang

Jc Mao

Jianming Liu

Brian Hutchinson

Sibel Yaman

Craig M Fisher

Chih-Kuan Yeh

George Edward Dahl

Deng Li

A method and apparatus comprising computer code configured to cause a processor or processors to receive an audio signal obtained from a microphone, input the audio signal into a neural-network based AEC model, and output an AEC signal from the neural-network based AEC model in which AEC is applied to the audio signal, and the AEC signal is a version of the audio signal in which acoustic echo noise of the audio signal is suppressed and target audio of the audio signal is sustained.

Kalmannet: a learnable Kalman filter for acoustic echo cancellation

A method for zero-shot event extraction, performed by a computer device. The method includes training a context encoder, a first definition encoder, and a second definition encoder with auto extracted context-definition alignment data; retrieving a plurality of verbal synsets from a lexical database; refining a representation model based on the context-definition alignment data and the plurality of verbal synsets; encoding a plurality of candidate event type definitions; encoding the refined representation model with the trained context encoder; and determining whether the encoded representation model belongs to one of the plurality of candidate event type definitions based on a cosine similarity between the encoded representation model, the trained context encoder, the first trained definition encoder, and the second trained definition encoder.

Efficient zero shot event extraction with context definition

A method for disentangled variational speech representation learning for voice conversion, performed by at least one processor, is provided. The method includes receiving input speech segments, encoding the input speech segments via a shared encoder to generate a speaker embedding and a content embedding, encoding the posterior distributions of the speaker embedding via a speaker encoder and encoding the posterior distributions of the content embedding via a content encoder to obtain encoded results, and decoding the encoded results by concatenating the encoded results to obtain a reconstructed speech output.

Techniques for disentangled variational speech representation learning for zero-shot voice conversion

A method including: receiving an input comprising natural language texts at an encoder; adding a token to the input; obtaining a last-layer hidden state as a natural language text representation; feeding the natural language text representation into a single-layer classification head; predicting a salience allocation based on the single-layer classification head; developing a salience-aware cross-attention (SACA) decoder to determine salience in the natural language text representation; mapping a plurality of salience degrees to a plurality of trainable salience embeddings; estimating an amount of signal to accept from the plurality of trainable salience embeddings; incorporating the salience allocation and the signal in a cross-attention layer model; and generating a summarization based on the SACA decoder and the cross-attention layer model.

Salience-aware cross-attention for abstractive summarization

A method of acoustic echo suppression using a recurrent neural network, performed by at least one processor, is provided. The method includes receiving a microphone signal and a far-end reference signal, estimating an echo suppressed signal and an echo signal based on the microphone signal and the far-end reference signal, estimating enhancement filters for the microphone signal based on the echo suppressed signal and the echo signal, generating an enhanced signal based on the enhancement filters, and adjusting the enhanced signal using automatic gain control (AGC) and outputting the adjusted signal.

Neuralecho: a self-attentive recurrent neural network for unified acoustic echo suppression, speaker aware speech enhancement and automatic gain control

A method, computer program, and computer system is provided for converting a singing voice of a first person associated with a first speaker to a singing voice of a second person using a speaking voice of the second person associated with a second speaker. A context associated with one or more phonemes corresponding to the singing voice of a first person is encoded, and the one or more phonemes are aligned to one or more target acoustic frames based on the encoded context. One or more mel-spectrogram features are recursively generated from the aligned phonemes, the target acoustic frames, and a sample of the speaking voice of the second person. A sample corresponding to the singing voice of a first person is converted to a sample corresponding to the second singing voice using the generated mel-spectrogram features.

Learning singing from speech

An apparatus and a method include receiving an input audio signal to be processed by a multi-band synchronized neural vocoder. The input audio signal is separated into a plurality of frequency bands. A plurality of audio signals corresponding to the plurality of frequency bands is obtained. Each of the audio signals is downsampled, and processed by the multi-band synchronized neural vocoder. An audio output signal is generated.

Multi-band synchronized neural vocoder

A method, computer program, and computer system is provided for identifying a speaker in at text based work. Labeled and unlabeled instances corresponding to one or more speakers are extracted. Pseudo-labels are inferred for the extracted unlabeled instances based on the labeled instances. One or more of the unlabeled instances are labeled based on the inferred pseudo-labels.

Extractive method for speaker identification in texts with self-training

A method, computer program, and computer system is provided for an all-deep-learning based AEC system by recurrent neural networks. The model consists of two stages, echo estimation stage and echo suppression stage, respectively. Two different schemes for echo estimation are presented herein: linear echo estimation by multi-tap filtering on far-end reference signal and non-linear echo estimation by single-tap masking on microphone signal. A microphone signal waveform and a far-end reference signal waveform are received. An echo signal waveform is estimated based on the microphone signal waveform and a far-end reference signal waveform. A near-end speech signal waveform is output based on subtracting the estimated echo signal waveform from the microphone signal waveform, and echoes are suppressed within the near-end speech signal waveform.

Neural-echo: an unified deep neural network model for acoustic echo cancellation and residual echo suppression

A model training method includes obtaining an image sample set and brief-prompt information; generating a content mask set according to the image sample set and the brief-prompt information; generating a to-be-trained image set according to the content mask set; obtaining, based on the image sample set and the to-be-trained image set, a predicted image set through a to-be-trained information synthesis model, the predicted image set comprising at least one predicted image, the predicted image being in correspondence to the image sample; and training, based on the predicted image set and the image sample set, the to-be-trained information synthesis model by using a target loss function, to obtain an information synthesis model.

Model training method, media information synthesis method, and related apparatuses

A method includes receiving, by a first encoder, an original speech segment, receiving, by a second encoder, an augmented speech segment of the original speech segment, generating, by the first encoder, a first speaker representation based on the original speech segment, generating, by the second encoder, a second speaker representation based on the augmented speech segment, and generating a contrastive loss based on the first speaker representation and the second speaker representation.

Progressive contrastive learning framework for self-supervised speaker verification

There is included a method and apparatus comprising computer code for instance-wise adaptive knowledge injection in a pre-trained language model (PTLM) including determining a necessity of external knowledge in a plurality of queries of a first dataset based on a likelihood that a respective query is solved by internal knowledge of a target model. Then, the one or more queries determined to need external knowledge may be augmented with pieces of external knowledge. A combined dataset may be generated by combining the first dataset and the one or more augmented queries, and the combined dataset may be applied to the target model.

Instance-level adaptive propulsion of external knowledge (IAPEK)

There is included a method and apparatus comprising computer code configured to cause a processor or processors to perform generating one or more aligned inventories, wherein the one or more aligned inventories are generated using one or more word sense inventories, obtaining a word in a context sentence, determining one or more semantic equivalence scores indicating semantic similarity between the word in the context sentence and each of one or more associated glosses in the one or more aligned inventories using a semantic equivalence recognizer model, and predicting a correct sense of the word in the context sentence based on the determined one or more semantic equivalence scores.

Bridging semantics between words and definitions via aligning word sense inventories

A method of end-to-end speaker diarization (EESD) using neural speaker clustering, performed by at least one processor, is provided. The method includes generating a set of speech labels corresponding to a set of speakers based on an input stream. The speech labels indicate whether dialogue of a speaker is speech or non-speech. The method further includes generating dialogue based on the set of speakers, extracting speaker embeddings from the dialogue, mapping the speaker embeddings to a cluster identification (ID), an overlapped speech value, or a non-speech value based on a neural network, and outputting EESD labels based on the mapping.

Techniques for end-to-end speaker diarization with generalized neural speaker clustering

An unsupervised text to speech system utilizing a lexicon to map input text to the phoneme sequence, which is expanded to the frame-level forced alignment with a speaker-dependent duration model. An alignment mapping module that converts the forced alignment to the unsupervised alignment (UA). Afterword, a Conditional Disentangled Sequential Variational Auto-encoder (C-DSVAE), serving as the self-supervised TTS AM, takes the predicted UA and a target speaker embedding to generate the mel spectrogram, which is ultimately converted to waveform with a neural vocoder.

Text to speech synthesis without using parallel text-audio data

Growing community of inventors

Kirkland, WA, United States of America

Dong Yu