Microsoft Technology Licensing, LLC

Tencent America LLC

Tencent Technology (shenzhen) Company Limited

Microsoft Corporation

Micriosoft Corporation

Dong Yu

Li Deng

Alejandro Acero

Chao Weng

Chengzhu Yu

Meng Yu

Heng Lu

Yifan Gong

Dan Su

Jia Cui

Jun Wang

Yun-Cheng Ju

Jinyu Li

James G Droppo

Ye-Yi Wang

Chunlei Zhang

Yong Xu

Gang Li

Michael Lewis Seltzer

Frank Torsten Bernd Seide

Guangsen Wang

Shi-Xiong Zhang

Jianshu Chen

Yanmin Qian

Dian Yu

Peidong Wang

Milind V Mahajan

Xiaodong He

Geoffrey G Zweig

Mei-Yuh Hwang

Xiaolong Li

Kaisheng N Yao

Peter K L Mau

Hongming Zhang

Xiaoyang Wang

Jie Chen

Jian Wu

Frank Seide

Kun Xu

Sangwoo Cho

Shi-xiong Zhang

Jiachen Lian

Peter Mau

Yan Zhang

Xuedong D Huang

Ruhi Sarikaya

Kuansan Wang

Li Qin Shen

Ying Shan

Julian James Odell

Xiao Li

Yonggen Ling

Asela J Gunawardana

Linfeng Song

Shan Yang

Yu Shrike Zhang

Adam Clyde Eversole

Xiang Li

Yue Yang

Haozhi Huang

John C Langford

Jian Jiao

Jiawei Li

Shixiong Zhang

Shiyin Kang

Anoop Deoras

Jian Xue

Lianwu Chen

Fei Wang

Xie Chen

Jui-Ting Huang

Alex Acero

Hang Su

Abdel-rahman Samir Abdel-rahman Mohamed

Shizhen Wang

Jianming Liu

Gregoire Mesnil

Brian Yan

Lifeng Jin

Thomas Ryan Hoens

Wenlin Yao

Haijing Wang

Xiaoman Pan

Jc Mao

Kaiqiang Song

Hao Zhang

Brian Hutchinson

George Edward Dahl

Chih-Kuan Yeh

Deng Li

Craig M Fisher

Sibel Yaman

Yu Zhang

Abdel-Rahman Samir Abdel-Rahman Mohamed

A method, apparatus, and non-transitory computer-readable medium for automatic speech recognition using conditional factorization for bilingual code-switched and monolingual speech may include receiving an audio observation sequence comprising a plurality of frames, the audio observation sequence including audio in a first language or a second language. The approach may further include mapping the audio observation sequence into a first sequence of hidden representations, the mapping being generated by a first encoder corresponding to the first language and mapping the audio observation sequence into a second sequence of hidden representations, the mapping being generated by a second encoder corresponding to the second language. The approach may further include generating a label-to-frame sequence based on the first sequence of hidden representations and the second sequence of hidden representations, using a joint neural network based model.

Conditional factorization for jointly modeling code-switched and monolingual ASR

A method performed by at least one processor includes receiving a first input stream of a task and a second input stream of a solution. The method further includes selecting the first input stream or the second input stream. The method further includes providing the selected input stream to an image conversion model and a language model. The method further includes creating, based on the selected input stream, a model ensemble of the conversion model and the language model. The method further includes outputting a prediction based on the model ensemble. The method may further include generating an image corresponding to text, converting a textual task into a multimodal task, and solving the multimodal task.

Method and apparatus for zero-shot natural language processing using visual imagination

A method and apparatus that identifies one or more characters within a text; determines one or more informative sections within the text, the one or more informative sections providing information regarding a gender of the one or more characters within the text; selects a most informative section from the one or more informative sections; extracts unlabeled instances corresponding to the gender of the one or more characters from the most informative section; iteratively trains a multi-task model using unlabeled corpora, the multi-task model performing both speaker identification and gender identification; and labels the gender of the one or more characters based on the extracted unlabeled instances and the multi-task model.

Multi-task self-training for character gender identification

A method, computer program, and computer system is provided for automated speech recognition. Audio data corresponding to one or more speakers is received. Covariance matrices of target speech and noise associated with the received audio data are estimated based on a gated recurrent unit-based network. A predicted target waveform corresponding to a target speaker from among the one or more speakers is generated by a minimum variance distortionless response function based on the estimated covariance matrices.

All deep learning minimum variance distortionless response beamformer for speech separation and enhancement

A method and apparatus comprising computer code configured to cause a processor or processors to receive an audio signal obtained from a microphone, input the audio signal into a neural-network based AEC model, and output an AEC signal from the neural-network based AEC model in which AEC is applied to the audio signal, and the AEC signal is a version of the audio signal in which acoustic echo noise of the audio signal is suppressed and target audio of the audio signal is sustained.

Kalmannet: a learnable Kalman filter for acoustic echo cancellation

A method for zero-shot event extraction, performed by a computer device. The method includes training a context encoder, a first definition encoder, and a second definition encoder with auto extracted context-definition alignment data; retrieving a plurality of verbal synsets from a lexical database; refining a representation model based on the context-definition alignment data and the plurality of verbal synsets; encoding a plurality of candidate event type definitions; encoding the refined representation model with the trained context encoder; and determining whether the encoded representation model belongs to one of the plurality of candidate event type definitions based on a cosine similarity between the encoded representation model, the trained context encoder, the first trained definition encoder, and the second trained definition encoder.

Efficient zero shot event extraction with context definition

A method for disentangled variational speech representation learning for voice conversion, performed by at least one processor, is provided. The method includes receiving input speech segments, encoding the input speech segments via a shared encoder to generate a speaker embedding and a content embedding, encoding the posterior distributions of the speaker embedding via a speaker encoder and encoding the posterior distributions of the content embedding via a content encoder to obtain encoded results, and decoding the encoded results by concatenating the encoded results to obtain a reconstructed speech output.

Techniques for disentangled variational speech representation learning for zero-shot voice conversion

A method including: receiving an input comprising natural language texts at an encoder; adding a token to the input; obtaining a last-layer hidden state as a natural language text representation; feeding the natural language text representation into a single-layer classification head; predicting a salience allocation based on the single-layer classification head; developing a salience-aware cross-attention (SACA) decoder to determine salience in the natural language text representation; mapping a plurality of salience degrees to a plurality of trainable salience embeddings; estimating an amount of signal to accept from the plurality of trainable salience embeddings; incorporating the salience allocation and the signal in a cross-attention layer model; and generating a summarization based on the SACA decoder and the cross-attention layer model.

Salience-aware cross-attention for abstractive summarization

A method of acoustic echo suppression using a recurrent neural network, performed by at least one processor, is provided. The method includes receiving a microphone signal and a far-end reference signal, estimating an echo suppressed signal and an echo signal based on the microphone signal and the far-end reference signal, estimating enhancement filters for the microphone signal based on the echo suppressed signal and the echo signal, generating an enhanced signal based on the enhancement filters, and adjusting the enhanced signal using automatic gain control (AGC) and outputting the adjusted signal.

Neuralecho: a self-attentive recurrent neural network for unified acoustic echo suppression, speaker aware speech enhancement and automatic gain control

A method, computer program, and computer system is provided for converting a singing voice of a first person associated with a first speaker to a singing voice of a second person using a speaking voice of the second person associated with a second speaker. A context associated with one or more phonemes corresponding to the singing voice of a first person is encoded, and the one or more phonemes are aligned to one or more target acoustic frames based on the encoded context. One or more mel-spectrogram features are recursively generated from the aligned phonemes, the target acoustic frames, and a sample of the speaking voice of the second person. A sample corresponding to the singing voice of a first person is converted to a sample corresponding to the second singing voice using the generated mel-spectrogram features.

Learning singing from speech

An apparatus and a method include receiving an input audio signal to be processed by a multi-band synchronized neural vocoder. The input audio signal is separated into a plurality of frequency bands. A plurality of audio signals corresponding to the plurality of frequency bands is obtained. Each of the audio signals is downsampled, and processed by the multi-band synchronized neural vocoder. An audio output signal is generated.

Multi-band synchronized neural vocoder

A method, computer program, and computer system is provided for identifying a speaker in at text based work. Labeled and unlabeled instances corresponding to one or more speakers are extracted. Pseudo-labels are inferred for the extracted unlabeled instances based on the labeled instances. One or more of the unlabeled instances are labeled based on the inferred pseudo-labels.

Extractive method for speaker identification in texts with self-training

A method, computer program, and computer system is provided for an all-deep-learning based AEC system by recurrent neural networks. The model consists of two stages, echo estimation stage and echo suppression stage, respectively. Two different schemes for echo estimation are presented herein: linear echo estimation by multi-tap filtering on far-end reference signal and non-linear echo estimation by single-tap masking on microphone signal. A microphone signal waveform and a far-end reference signal waveform are received. An echo signal waveform is estimated based on the microphone signal waveform and a far-end reference signal waveform. A near-end speech signal waveform is output based on subtracting the estimated echo signal waveform from the microphone signal waveform, and echoes are suppressed within the near-end speech signal waveform.

Neural-echo: an unified deep neural network model for acoustic echo cancellation and residual echo suppression

A model training method includes obtaining an image sample set and brief-prompt information; generating a content mask set according to the image sample set and the brief-prompt information; generating a to-be-trained image set according to the content mask set; obtaining, based on the image sample set and the to-be-trained image set, a predicted image set through a to-be-trained information synthesis model, the predicted image set comprising at least one predicted image, the predicted image being in correspondence to the image sample; and training, based on the predicted image set and the image sample set, the to-be-trained information synthesis model by using a target loss function, to obtain an information synthesis model.

Model training method, media information synthesis method, and related apparatuses

A method includes receiving, by a first encoder, an original speech segment, receiving, by a second encoder, an augmented speech segment of the original speech segment, generating, by the first encoder, a first speaker representation based on the original speech segment, generating, by the second encoder, a second speaker representation based on the augmented speech segment, and generating a contrastive loss based on the first speaker representation and the second speaker representation.

Progressive contrastive learning framework for self-supervised speaker verification

Growing community of inventors

Kirkland, WA, United States of America

Dong Yu