Microsoft Technology Licensing, LLC

Senthil Kumar Velayutham

Ross Garrett Cutler

Sriram Srinivasan

Ankit Tandon

Humayun Mukhtar Khan

Rajneesh Mahajan

Arash Ghanaie-Sichanie

Ramin Mehran

Karlton David Sequeira

Robert Aichner

Subhashri Iyer

Jayant Ajit Gupchup

Guo-Wei Sheih

Lon-Chan Chu

William George Verthein

Jian Wu

Srivatsa K Srinivasan

Mathrubootham Janakiraman

Nicholas David Burton

Jian Wang

Xinding Sun

Qi Liu

Kyle A Larsen

Armen Hovhannisyan

Srivatsa K Srinivisan

A system and method for selecting a target device out of a larger group of candidate devices for rendering a response from a virtual assistant to an end-user is disclosed. The system determines that a same trigger phrase included in an utterance has been received by multiple devices that are in proximity to one another at around the same time. These candidate devices can collect attention data, such as user gaze toward a device, to select the device that was most likely the intended recipient of the utterance. The system is configured to control the virtual assistant to render a response solely via the selected device.

Selective response rendering for virtual assistants

Disclosed in some examples, are methods, systems, and machine-readable mediums for preventing unintended activation of voice command processing of a voice activated device. A first audio signal may be an audio signal that is to be output to a speaker communicatively coupled to the computing device. A second audio signal may be input from a microphone or other audio capture device. Both audio signals are input to a keyword detector to check for the presence of activation keywords. If the activation keyword(s) are detected in the second audio signal but not the first audio signal the voice command processing of the device is activated as this is likely a command from the user and not feedback from the loudspeaker.

Audio pipeline for simultaneous keyword spotting, transcription, and real time communications

Disclosed in some examples are devices, systems, methods, and machine-readable mediums which provide for a network-based communication service that allows users in a network-based communication session to subscribe to video streams of the communication session that are of interest by subscribing to content features of those video streams rather than the video streams themselves. Content features may be any object in the video stream that is recognized by applying an image processing algorithm to one or more video frames. Example content features may include recognized users and/or other recognized objects.

Content feature based video stream subscriptions

A device for communicating with a remote device is disclosed, which includes a processor and a memory in communication with the processor. The memory includes executable instructions that, when executed, cause the processor to control the device to perform functions of establishing, via a communication network, a communication session with the remote device; capturing a speech spoken by a user and generating audio data representing the captured speech by the user; encoding the audio data for transmission to the remote device via the communication network; converting the audio data to text data representing the captured speech; and transmitting, during the communication session, the encoded audio data and the text data to the remote device via the communication network. The device thus can provide the text data representing the captured speech when a quality of the encoded audio signal received by the remote device is below a predetermined level.

Artificially generated speech for a communication session

Provided herein are systems, methods, and software for facilitating a video environment. In at least one implementation, video of at least a first object to be displayed in the video environment is received. Video of at least a second object is also received which is to be displayed with the first object in the video environment. A zoom operation is identified to be performed on the first object to reduce a difference between a size of the first object and a size of the second object. The zoom operation is then performed on the video of the first object. In response to the zoom operation, the first object and the second object are displayed in the video environment.

Video display modification for video environments

Content is received at a receiving equipment from a transmitting user terminal over a network in a communication session between a transmitting user and a receiving user. The received content comprises audio data representing speech spoken by a voice of the transmitting user, and further comprises text data generated from speech spoken by the voice of the transmitting user during the communication session. At the receiving equipment, at least a portion of the received text data is converted to artificially-generated audible speech based on a model of the transmitting user's voice stored at the receiving equipment (and in embodiments in dependence on the receive audio quality). The received audio data and the artificially-generated speech are supplied to be played out to the receiving user through one or more speakers.

Provided herein are systems, methods, and software for facilitating a video conference environment. In at least one implementation, video captured of a scene is received. A modification to display of the video is identified based at least in part on a share of a display space associated with an object in the scene relative to a share of another display space associated with another object in another scene. The modification to the display of the video is then initiated. In another implementation, the modification may also be based on a target share indicated by way of an interactive graphic.

Video display modification for video conferencing environments

Provided herein are systems, methods, and software for facilitating collaboration environments. In at least one implementation, a presentation of an initial view of a collaboration scene is presented within a collaboration environment. A portion of the collaboration scene is identified that corresponds to an object of interest selected for separate viewing. A separate presentation within the collaboration environment is initiated related to the portion of the scene corresponding to the object of interest.

Collaboration environments and views

Real-time media optimization may be provided. First, a remote session may be established with a remote computing device. Then, during the remote session, non-real-time media data may be exchanged with the remote computing device over a server path. Moreover, real-time media data may be exchanged with the remote computing device over a media path during the remote session.

Real-time media optimization over remoted sessions

Frames containing audio data may be received, the audio data having been derived from a microphone array, at least some of the frames containing residual acoustic echo after having acoustic echo partially removed therefrom. Probability distribution functions are determined from the frames of audio data. A probability distribution function comprises likelihoods that respective directions are directions of sources of sounds. An active speaker may be identified in frames of video data based on the video data and based on audio information derived from the audio data, where use of the audio information as a basis for identifying the active speaker is controlled by determining whether the probability distribution functions indicate that corresponding audio data includes residual acoustic echo.

Identifying far-end sound

In a network multimedia conference, a bridge service that functions as a proxy for a client to join the conference dynamically selects a video stream from multiple video streams originating from the other participants of the conference and sends the selected video stream to the client for viewing. The bridge service monitors conferencing events of each participant and updates conferencing activity state variables of the participant, such as whether the participant is being shown to the client, whether the participant is talking, how long the participant has been shown or not shown, etc. Periodically, a weight assigned to each of the participants is computed based on the state variables, and the video stream of the participant with the highest weight is then selected for viewing by the client.

Method and system for selectively viewing participants of a multimedia network conference

Performing conferencing without requiring communication using a specific connection port. A software module receives a function call from a conferencing application. The function call may include high-level instructions to perform a basic conferencing function, such as the communication of data from a specified buffer area. The software module then performs all of the details necessary in order to establish and maintain a connection with a conferencing server (or with one or more other participants in the case of peer-to-peer conferencing). While the conferencing application may be configured to optionally perform teleconferencing using a particular conferencing protocol and a particular conferencing connection port without the use of the software module, the software module may also be optionally used to establish, maintain, and exchanging conferencing information over an entirely different port when, for example, the usual conferencing connection port is not available.

Growing community of inventors

Sammamish, WA, United States of America

Senthil Kumar Velayutham