Microsoft Technology Licensing, LLC

Elwha LLC

Other

Micrsoft Corporation

Microsoft Corporation

Esposito, Mario

Xuedong D Huang

Roderick A Hyde

Clarence T Tegreene

Nathan P Myhrvold

Mark A Malamud

Royce A Levien

Marc Eliot Davis

Matthew Graham Dyor

Robert W Lord

Richard T Lord

William Henry Gates, Iii

Lowell L Wood, Jr

Edward K Y Jung

Daniel Reed

Jordin T Kare

Alejandro Acero

Eric J Horvitz

Li Deng

Joshua T Goodman

Zhengyou Zhang

Zheng Chen

Zicheng Liu

Gary William Flake

Kuansan Wang

Susan Theresa Dumais

Michael Jack Sinclair

Christopher A Meek

Raymond Edward Ozzie

Kamal Jain

James G Droppo

Anoop Gupta

Lili Cheng

Jianfeng Gao

Shipeng Li

Heung-Yeung Shum

Alexander G Gounares

Jin Li

Dong Yu

Dong Li

Christian F Huitema

John Carlton Platt

Philip Andrew Chou

Yong Rui

Michel Pahud

Cha Zhang

William B Dolan

Yun-Cheng Ju

Eric David Brill

Li Jiang

Qi Lu

Satya Nadella

Harry Shum

Milind V Mahajan

Nanshan Zeng

Lijuan Qin

Zhuo Chen

Takuya Yoshioka

Dimitrios Basile Dimitriadis

Daniel A Gerrity

Andreas Stolcke

Ye-Yi Wang

Hsiao-Wuen Hon

Mei-Yuh Hwang

Wei Xiong

Oliver Hurst-Hiller

Michael D Plumpe

Arul A Menezes

Fileno A Alleva

Danny Allen Reed

Olivier Nano

Trenholme J Griffin

Adi Diamant

Tanvi Surti

Eyal Krupka

Yu Shi

Jonathan Jay Cadiz

Jian-Tao Sun

Hagai Thomas Attias

Frank Torsten Bernd Seide

Amit Kumar Agarwal

Michael J Rozak

Todd D Newman

Yuan Kong

Christopher Brian Quirk

Jai Srinivasan

Shuangyu Chang

Doug Burger

Ramez Naam

Christopher John Brockett

Gursharan Singh Sidhu

Weizhu Chen

Hongjiang Zhang

Silviu-Petru Cucerzan

Rajesh Hegde

Kyle G Peltonen

Stelios Paparizos

David J Kurlander

Jian-Lai Zhou

A computer implemented method processes audio streams recorded during a meeting by a plurality of distributed devices. Operations include performing speech recognition on each audio stream by a corresponding speech recognition system to generate utterance-level posterior probabilities as hypotheses for each audio stream, aligning the hypotheses and formatting them as word confusion networks with associated word-level posteriors probabilities, performing speaker recognition on each audio stream by a speaker identification algorithm that generates a stream of speaker-attributed word hypotheses, formatting speaker hypotheses with associated speaker label posterior probabilities and speaker-attributed hypotheses for each audio stream as a speaker confusion network, aligning the word and speaker confusion networks from all audio streams to each other to merge the posterior probabilities and align word and speaker labels, and creating a best speaker-attributed word transcript by selecting the sequence of word and speaker labels with the highest posterior probabilities.

Speaker attributed transcript generation

Examples of array geometry agnostic multi-channel personalized speech enhancement (PSE) extract speaker embeddings, which represent acoustic characteristics of one or more target speakers, from target speaker enrollment data. Spatial features (e.g., inter-channel phase difference) are extracted from input audio captured by a microphone array. The input audio includes a mixture of speech data of the target speaker(s) and one or more interfering speaker(s). The input audio, the extracted speaker embeddings, and the extracted spatial features are provided to a trained geometry-agnostic PSE model. Output data is produced, which comprises estimated clean speech data of the target speaker(s) that has a reduction (or elimination) of speech data of the interfering speaker(s), without the trained PSE model requiring geometry information for the microphone array.

Array geometry agnostic multi-channel personalized speech enhancement

The disclosure herein describes training a global model based on a plurality of data sets. The global model is applied to each data set of the plurality of data sets and a plurality of gradients is generated based on that application. At least one gradient quality metric is determined for each gradient of the plurality of gradients. Based on the determined gradient quality metrics of the plurality of gradients, a plurality of weight factors is calculated. The plurality of gradients is transformed into a plurality of weighted gradients based on the calculated plurality of weight factors and a global gradient is generated based on the plurality of weighted gradients. The global model is updated based on the global gradient, wherein the updated global model, when applied to a data set, performs a task based on the data set and provides model output based on performing the task.

Dynamic gradient aggregation for training neural networks

A computer implemented method includes receiving audio signals representative of speech via multiple audio streams transmitted from corresponding multiple distributed devices, performing, via a neural network model, continuous speech separation for one or more of the received audio signals having overlapped speech, and providing the separated speech on a fixed number of separate output audio channels.

Processing overlapping speech from distributed devices

Systems, methods, and computer-readable storage media are provided for conducting searches utilizing search navigation patterns. Search queries are received that include search terms that are of a particular type. It is recognized that at least one prior search session has been conducted that included a search query having search terms of an equivalent or similar type and followed a particular navigation pattern. Such prior search(es) may have been conducted by the user or by a different user and/or may have a navigation pattern that was affirmatively recorded by the requesting user or that was recorded by the system without explicit contemporaneous user instruction to do so. Upon identifying the navigation pattern associated with the prior search, the system effectively conducts a search session following the navigation pattern.

Conducting search sessions utilizing navigation patterns

Systems, methods, and computer-readable storage devices are disclosed for generating smart notes for a meeting based on participant actions and machine learning. One method including: receiving meeting data from a plurality of participant devices participating in an online meeting; continuously generating text data based on the received audio data from each participant device of the plurality of participant devices; iteratively performing the following steps until receiving meeting data for the meeting has ended, the steps including: receiving an indication that a predefined action has occurred on the first participating device; generating a participant segment of the meeting data for at least the first participant device from a first predetermined time before when the predefined action occurred to when the predefined action occurred; determining whether the receiving meeting data of the meeting has ended; and generating a summary of the meeting.

Systems, methods, and computer-readable storage device for generating notes for a meeting based on participant actions and machine learning

A transcription of audio speech included in electronic content associated with a meeting is created by an ASR model trained on speech-to-text data. The transcription is post-processed by modifying text included in the transcription, for example, by modifying punctuation, grammar, or formatting introduced by the ASR model and by changing or omitting one or more words that were included in both the audio speech and the transcription. After the transcription is post-processed, output based on the post-processed transcription is generated in the form of a meeting summary and/or template.

Automated meeting minutes generator

Embodiments are provided for securing data access to machine learning training data at a plurality of distributed computing devices. Electronic content including original data that corresponds to a preferred data security level is divided into a plurality of microsegments. The plurality of microsegments is restrictively distributed to a plurality of computing devices which apply transcription labels to the plurality of microsegments. The labeled microsegments are reconstructed into training data which is then used to train a machine learning model while facilitating an improvement in data security of the original data included with the training data from the reconstructed microsegments.

Microsegment secure speech transcription

A computer implemented method includes receiving information streams on a meeting server from a set of multiple distributed devices included in a meeting, receiving audio signals representative of speech by at least two users in at least two of the information streams, receiving at least one video signal of at least one user in the information streams, associating a specific user with speech in the received audio signals as a function of the received audio and video signals, and generating a transcript of the meeting with an indication of the specific user associated with the speech.

Audio-visual diarization to identify meeting attendees

A method for facilitating a remote conference includes receiving a digital video and a computer-readable audio signal. A face recognition machine is operated to recognize a face of a first conference participant in the digital video, and a speech recognition machine is operated to translate the computer-readable audio signal into a first text. An attribution machine attributes the text to the first conference participant. A second computer-readable audio signal is processed similarly, to obtain a second text attributed to a second conference participant. A transcription machine automatically creates a transcript including the first text attributed to the first conference participant and the second text attributed to the second conference participant.

Computerized intelligent assistant for conferences

Systems and methods may be used to provide transcription and translation services. A method may include initializing a plurality of user devices with respective language output selections in a translation group by receiving a shared identifier from the plurality of user devices and transcribing the audio stream to transcribed text. The method may include translating the transcribed text to one or more of the respective language output selections when an original language of the transcribed text differs from the one or more of the respective language output selections. The method may include sending, a user device in the translation group, the transcribed text including translated text in a language corresponding to the respective language output selection for the user device. In an example, the method may include customizing the transcription or the translation, such as to a particular topic, location, user, or the like.

Customized transcribed conversations

A computer implemented method includes receiving audio streams at a meeting server from two distributed devices that are streaming audio captured during an ad-hoc meeting between at least two users, comparing the received audio streams to determine that the received audio streams are representative of sound from the ad-hoc meeting, generating a meeting instance to process the audio streams in response to the comparing determining that the audio streams are representative of sound from the ad-hoc meeting, and processing the received audio streams to generate a transcript of the ad-hoc meeting.

Distributed device meeting initiation

Growing community of inventors

Woodinville, WA, United States of America

Xuedong D Huang