Microsoft Technology Licensing, LLC

Amazon Technologies, Inc.

Micorsoft Corpration

Microsoft Corporation

James G Droppo

Alejandro Acero

Li Deng

Michael Lewis Seltzer

Dong Yu

Zhengyou Zhang

Zicheng Liu

Xuedong D Huang

Chao Wang

Chao Weng

Ming Sun

Qingming Tang

Qin Zhang

Ivan Jelev Tashev

Ariya Rastrow

Sumit Garg

Chieh-Chi Kao

Amarnag Subramanya

Rong Chen

Chia-Jung Chang

Luis Buera

Pegah Ghahremani

Henrique Sarmento Malvar

Michael Jack Sinclair

Rui Zhao

Jinyu Li

Yifan Gong

Gang Li

Yun-Cheng Ju

Yang Liu

Milind V Mahajan

Bjorn Hoffmeister

Geoffrey G Zweig

Oguz Hasan Elibol

Frank Torsten Bernd Seide

Yu Shrike Zhang

Eli Joshua Fidler

Jian Wu

Viktor Rozgic

Roberto Barra Chicote

John C Langford

Andrew Smith

Guoli Ye

Roland Maximilian Rolf Maas

Gautam Tiwari

Zeya Chen

Venkata Kishore Nandury

Veerdhawal Pande

Andrew Oberlin

Xuedong David Huang

Yixiong Meng

Pankaj Sitpure

Jie Liang

Steve Mark Lorusso

Gurpreet Singh Chadha

Maarten Van Segbroeck

Gopinath Chennupati

Milind Mukesh Rao

Santosh Kumar Patro

Buddha Puneeth Nandanoor

Anand Mohan

Prahalad Venkataramanan

Andrew Thomas Bydlon

Sripal Mehta

Chia-Hao Chang

Grzegorz Beringer

Constantinos Boulis

Yanli Zheng

Patrick An-Phu Nguyen

Xing Fan

Ozlem Kalinli

Avner May

Yu Zhang

Amarnag c/o Microsoft Corporation Subramanya

A first neural network (NN) model may generate labels for training a second NN model. The second NN model may represent instances of a NN model operating on multiple different devices (e.g., decentralized user and/or edge devices). The system may include using a 'teacher' model to process data received by one or more of the devices to generate a labeled dataset. The system may use the labeled dataset and a “student” model to calculate gradient data for updating the student model. The student model may be the same or similar to NN model instances operating on the devices. The system may validate the updated student model to determine, for example, whether it exhibits improved performance when processing the newly received data and/or historical data. The system may distribute the validated update to the devices.

Continuous learning for machine learning models

A system may include a first acoustic event detection (AED) component configured to detect a predetermined set of acoustic events, and include a second AED component configured to detect custom acoustic events that a user configures a device to detect. The first and second AED components are configured to perform task-specific processing, and may receive as input the same acoustic feature data corresponding to audio data that potentially represents occurrence of one or more events. Based on processing by the first and second AED components, a device may output data indicating that one or more acoustic events occurred, where the acoustic events may be a predetermined acoustic event and/or a custom acoustic event.

Acoustic event detection

A speech-processing system may determine potential endpoints in a user's speech. Such endpoint prediction may include determining a potential endpoint in a stream of audio data, and may additionally including determining an endpoint score representing a likelihood that the potential endpoint represents an end of speech representing a complete user input. When the potential endpoint has been determined, the system may publish a transcript of speech that preceded the potential endpoint, and send it to downstream components. The system may continue to transcribe audio data and determine additional potential endpoints while the downstream components process the transcript. The downstream components may determine whether the transcript is complete; e.g., represents the entirety of the user input. Final endpoint determinations may be made based on the results of the downstream processing including automatic speech recognition, natural language understanding, etc.

Endpointing in speech processing

A system may be configured to detect custom acoustic events, where the system generates an acoustic event profile for the custom acoustic event based on a natural language description provided by a user and using an audio sample of the described acoustic event. For example, the user may describe the custom acoustic event as 'dog bark.' The system may ask the user questions to refine the description (e.g., dog breed, dog gender, age, etc.). Using an audio sample of the refined description, the system may then determine that audio captured in the user's environment is a potential sample of the custom acoustic event. Such captured audio may be presented to the user for confirmation, and then may be used to detect future occurrences of the custom acoustic event in the user's environment.

A speech-processing system receives input data representing text. A first encoder processes segments of the text to determine embedding data representing the text, and a second encoder processes corresponding audio data to determine prosodic data corresponding to the text. The embedding and prosodic data is processed to create output data including a representation of speech corresponding to the text and prosody.

Synthetic speech processing

Techniques and constructs can reduce the time required to determine solutions to optimization problems such as training of neural networks. Modifications to a computational model can be determined by a plurality of nodes operating in parallel. Quantized modification values can be transmitted between the nodes to reduce the volume of data to be transferred. The quantized values can be as small as one bit each. Quantization-error values can be stored and used in quantizing subsequent modifications. The nodes can operate in parallel and overlap computation and data transfer to further reduce the time required to determine solutions. The quantized values can be partitioned and each node can aggregate values for a corresponding partition.

Computing system for training neural networks

A neural network is structured with a plurality of levels of nodes. Each level has a level-specific stabilization parameter that adjusts a learning rate, at a corresponding level, during training. The stabilization parameter has a value that varies inversely relative to a change in an objective training function during back-propagation of the error through the level.

Self-stabilized deep neural network

Non-limiting examples of the present disclosure describe advancements in acoustic-to-word modeling that improve accuracy in speech recognition processing through the replacement of out-of-vocabulary (OOV) tokens. During the decoding of speech signals, better accuracy in speech recognition processing is achieved through training and implementation of multiple different solutions that present enhanced speech recognition models. In one example, a hybrid neural network model for speech recognition processing combines a word-based neural network model as a primary model and a character-based neural network model as an auxiliary model. The primary word-based model emits a word sequence, and an output of character-based auxiliary model is consulted at a segment where the word-based model emits an OOV token. In another example, a mixed unit speech recognition model is developed and trained to generate a mixed word and character sequence during decoding of a speech signal without requiring generation of OOV tokens.

Advancing word-based speech recognition processing

A neural network is structured to connect the input values of an input set, at each level, to that level's output using a linear bypass connection. The linear bypass connection passes the input values, to the output, without applying a non-linear function to them.

Linearly augmented neural network

Various systems and methods for multi-talker speech separation and recognition are disclosed herein. In one example, a system includes a memory and a processor to process mixed speech audio received from a microphone. In an example, the processor can also separate the mixed speech audio using permutation invariant training, wherein a criterion of the permutation invariant training is defined on an utterance of the mixed speech audio. In an example, the processor can also generate a plurality of separated streams for submission to a speech decoder.

Multi-talker speech recognizer

A sequence recognition system comprises a prediction component configured to receive a set of observed features from a signal to be recognized and to output a prediction output indicative of a predicted recognition based on the set of observed features. The sequence recognition system also comprises a classification component configured to receive the prediction output and to output a label indicative of recognition of the signal based on the prediction output.

Prediction-based sequence recognition

The claimed subject matter includes a system and method for recognizing mixed speech from a source. The method includes training a first neural network to recognize the speech signal spoken by the speaker with a higher level of a speech characteristic from a mixed speech sample. The method also includes training a second neural network to recognize the speech signal spoken by the speaker with a lower level of the speech characteristic from the mixed speech sample. Additionally, the method includes decoding the mixed speech sample with the first neural network and the second neural network by optimizing the joint likelihood of observing the two speech signals considering the probability that a specific frame is a switching point of the speech characteristic.

Mixed speech recognition

Growing community of inventors

Duvall, WA, United States of America

James G Droppo