Microsoft Technology Licensing, LLC

Meta Platforms, Inc.

Sriram Srinivasan

Soren Skak Jensen

Koen Bernard Vos

Ming-Chieh Lee

Kenneth Martin Tubbs

Senthil Kumar Velayutham

Vinod Prakash

Ross Garrett Cutler

Michael John Novak

Yan Lu

Brian R Meyers

Wei Xiong

Bin Wang

Mu Han

Arash Ghanaie-Sichanie

XiangYu Kong

Chani Ayn Doggett

David Matthew Kruse

Abolade Gbadegesin

Colleen Elizabeth Hamilton

Xiulian Peng

Ramin Mehran

Karlton David Sequeira

Jayant Ajit Gupchup

Kiran Tatapudi

Shawn Crispin Wright

Anshul Rawat

David Yuheng Zhao

Ryan S Menezes

Brian Scott Aust

Thomas Ewan Jolly

Daniel Oliver

John Edward Gallardo

Bartosz Henryk Paliswiat

Karsten Vandborg Sørensen

Shawn Patrick Henry

Hongwu Huai

Yisheng Yao

Charles Aaron Hare

Yuchen Huang

Mykola Lando

Ajitesh Chandak

Mahmut Arslan

Senthil Velayutham

Colleen E Hamilton

Karsten Vandborg SØRENSEN

Apparatus and methods are disclosed for using machine learning models with private and public domains. Operations can be applied to transform input to a machine learning model in a private domain that is kept secret or otherwise made unavailable to third parties. In one example of the disclosed technology, a method includes applying a private transform to produce transformed input, providing the transformed input to a machine learning model that was trained using a training set modified by the private transform, and generating inferences with the machine learning model using the transformed input. Examples of suitable transforms that can be employed include matrix multiplication, time or spatial domain to frequency domains, and partitioning a neural network model such that an input and at least one hidden layer form part of the private domain, while the remaining layers form part of the public domain.

Protecting deep learned models

Systems and methods for social audio streaming may include (1) configuring, for a user participating in a media call with a group of additional users, a virtual visual surface that includes (i) an on-screen media call interface, presented within a viewable area of a display element of a device of the user, with a first set of user tiles, and (ii) an off-screen media call interface, positioned outside of the viewable area in a lateral direction relative to the viewable area, with a second set of user tiles and (2) spatializing an audio stream of an additional user, from the group of additional users, based on a position of the additional user's user tile within the virtual visual surface. Various other methods, systems, and computer-readable media are also disclosed.

Social audio streaming

Innovations in phase quantization during speech encoding and phase reconstruction during speech decoding are described. For example, to encode a set of phase values, a speech encoder omits higher-frequency phase values and/or represents at least some of the phase values as a weighted sum of basis functions. Or, as another example, to decode a set of phase values, a speech decoder reconstructs at least some of the phase values using a weighted sum of basis functions and/or reconstructs lower-frequency phase values then uses at least some of the lower-frequency phase values to synthesize higher-frequency phase values. In many cases, the innovations improve the performance of a speech codec in low bitrate scenarios, even when encoded data is delivered over a network that suffers from insufficient bandwidth or transmission quality problems.

Phase reconstruction in a speech decoder

Disclosed in some examples are methods, systems, and machine-readable mediums which determine jitter buffer delay by inputting jitter buffer and currently observed network status information to a machine learned model that is trained using a reinforcement learning (RL) method. The model maps these inputs to an action to compress, stretch, or hold the jitter buffer delay, which is used by a recipient computing device to optimize the jitter buffer delay. The model may be trained using a simulator that uses network traces of past real streaming sessions (e.g., communication sessions) of users. By training the model through reinforcement learning, the model learns to make better decisions through reinforcement in the form of reward signals that reflect the performance of each decision.

Reinforcement learning for jitter buffer control

Techniques for prioritization of data streams include transmitting a plurality of data streams to a first device over a network where each data stream is configured to cause the first device to perform a specific function, dynamically detecting a bandwidth limitation limiting an amount of available bandwidth for transmitting the plurality of data streams, dynamically determining bandwidth reduction criteria by identifying a first subset of the plurality of data streams to stop transmitting based on the specific function that each respective data stream causes the first device to perform, temporarily stopping transmitting of the first subset of data streams configured to cause the first device to perform a first set of functions, and continuing to transmit a second subset of data streams not included in the first subset of the data streams to the first device to cause the first device to continue to perform a second set of functions.

Throttling and prioritization of multiple data streams

Disclosed in some examples, are methods, systems, and machine-readable mediums for preventing unintended activation of voice command processing of a voice activated device. A first audio signal may be an audio signal that is to be output to a speaker communicatively coupled to the computing device. A second audio signal may be input from a microphone or other audio capture device. Both audio signals are input to a keyword detector to check for the presence of activation keywords. If the activation keyword(s) are detected in the second audio signal but not the first audio signal the voice command processing of the device is activated as this is likely a command from the user and not feedback from the loudspeaker.

Audio pipeline for simultaneous keyword spotting, transcription, and real time communications

A device for communicating with a remote device is disclosed, which includes a processor and a memory in communication with the processor. The memory includes executable instructions that, when executed, cause the processor to control the device to perform functions of establishing, via a communication network, a communication session with the remote device; capturing a speech spoken by a user and generating audio data representing the captured speech by the user; encoding the audio data for transmission to the remote device via the communication network; converting the audio data to text data representing the captured speech; and transmitting, during the communication session, the encoded audio data and the text data to the remote device via the communication network. The device thus can provide the text data representing the captured speech when a quality of the encoded audio signal received by the remote device is below a predetermined level.

Artificially generated speech for a communication session

Technology related to determining a user engagement with software programs is disclosed. In one example of the disclosed technology, a method can include receiving a plurality of signals indicating states of the computer and a software application executing on the computer. The method can include combining the signals to determine a user engagement with the software application. The method can include storing a user engagement log based on the determined user engagement with the software application.

Determining user engagement with software applications

Phase quantization in a speech encoder

Techniques are described for managing synchronized jitter buffers for streaming data (e.g., for real-time audio and/or video communications). A separate jitter buffer can be maintained for each codec. For example, as data is received in network packets, the data is added to the jitter buffer corresponding to the codec that is associated with the received data. When data needs to be read, the same amount of data is read from each of the jitter buffers. In other words, at each instance where data needs to be obtained (e.g., for decoding and playback), the same amount of data is obtained from each of the jitter buffers. In addition, the multiple jitter buffers use the same playout timestamp that is synchronized across the multiple of jitter buffers.

Synchronized jitter buffers to handle codec switches

Growing community of inventors

Sammamish, WA, United States of America

Sriram Srinivasan