Microsoft Technology Licensing, LLC

Brian C McDowell

Stephen Scott Evans

Robert Norman Heitkamp

Philip Andrew Edry

Steven Wilssens

Ziyad Ibrahim

Nadim Y Abdo

Darren Alexander Apfel

Paul Lee Coleman, Iv

David Douglas Bohn

Stephen G Latta

Rod G Fleck

Jeffrey Alan Kohler

Paolo V Malabuyo

Julia Louise Meinershagen

Mark Jeffrey Weinberg

Sebastian Sylvan

Paul J Radek

Owen W Paulus

Jeff Cole

Ivann Alvarado

David J Jurenka

The techniques disclosed herein provide apparatuses and related methods for the communication of spatial audio and related metadata. In some implementations, a source provides prerecorded spatial audio that has embedded metadata. A computing device processes the prerecorded spatial audio to generate an audio codec that is segmented to include a first section of audio data and a second section that includes metadata extracted from the prerecorded spatial audio. The generated audio codec may be received by a device that includes an encoder. The encoder may process the generated audio codec to generate audio data that includes the metadata.

Frame coding for spatial audio data

Methods and devices for correcting warping in spatial audio may include identifying a geometric transform that defines a geometric warping between a first spatial geometric model that represents how sound is produced in a first volumetric space and a second spatial geometric model that represents how sound is produced in a second volumetric space different from the first volumetric space. The methods and devices may include determining an inverse of the geometric transform that compensates for the geometric transform. The methods and devices may include applying the inverse of the geometric transform to a first location in the first spatial geometric model by mapping the first location to a second location in the second spatial geometric model to correct for the geometric warping.

Spatial audio warp compensator

A system for enabling a shared three-dimensional ('3D') audio bed available to multiple software applications is provided. The system manages bed metadata defining a number of speaker objects of a 3D audio bed. The bed metadata also associates each speaker object with a location, which in some configurations, is defined by a three-dimensional coordinate system. The bed metadata is communicated to a plurality of applications. The applications can then generate custom 3D audio data that associates individual audio streams with individual speaker objects of the 3D audio bed. The applications can then communicate the custom 3D audio data to a 3D audio bed engine, which causes the processing and rendering of the custom 3D audio data to an output device utilizing a selected spatialization technology. Aspects of the 3D bed can be altered when the spatialization technology or the output device changes.

Shared three-dimensional audio bed

A system for automatically tracking movement of a user participating in a video chat application executing in a computing device is disclosed. A capture device connected to the computing device captures a user in a field of view of the capture device and identifies a sub-frame of pixels identifying a position of the head, neck and shoulders of the user in a capture frame of a capture area. The sub-frame of pixels is displayed to a remote user at a remote computing system who is participating in the video chat application with the user. The capture device automatically tracks the position of the head, neck and shoulders of the user as the user moves to a next location within the capture area. A next sub-frame of pixels identifying a position of the head, neck and shoulders of the user in the next location is identified and displayed to the remote user at the remote computing device.

Automatically tracking user movement in a video chat application

A system for intelligently determining replay locations in a multimedia content stream based on identifying events in the multimedia content stream is provided. In one embodiment, events in the multimedia content stream are identified by analyzing information in the multimedia content stream, in real time. In another embodiment, events in the multimedia content stream are identified by analyzing the viewing behavior and an emotional response from users viewing the multimedia content, in real time. One or more replay locations in the multimedia content stream are determined based on the events identified in the multimedia content stream. The multimedia content stream with the replay locations are displayed to a user via a user interface in the user's processing device.

Intelligent determination of replays based on event identification

A blocking image generating system and related methods include a head-mounted display device having an opacity layer. A method may include receiving a virtual image to be presented by display optics in the head-mounted display device. Lighting information and an eye-position parameter may be received from an optical sensor system in the head-mounted display device. A blocking image may be generated in the opacity layer of the head-mounted display device based on the lighting information and the virtual image. The location of the blocking image in the opacity layer may be adjusted based on the eye-position parameter.

Display with blocking image generation

Embodiments herein provide for a pre-processing stage or mechanism for enhancing data compression of an image by splitting or separating the color channels for pixels into planar data sources. Typically, image data is displayed as an interleaved set of channel types for a color space such that each pixel within an image (such as a bitmap) has one component for each color channel. By separating the interleaved bitmap into a planar representation, each plane can be compressed independently and often times achieve higher compression ratios. This is especially true for channels that are expected to stay fairly constant throughout the bitmap most of the time. The utilization of the above embodiment can be further enhanced through manipulation of channel data or through converting the color space from one form to another.

Pre-processing of image data for enhanced compression

Streaming content customization techniques are described. In an implementation, metadata, that is associated with an element, is synchronized with a stream of content that includes the element. The element is configured to accept an input to access related content that provides context for a scene that includes the element. After the related content is output, playback of the stream of content is resumed from the point at which the input was accepted to access the related content.

Customizing streaming content presentation

Frames of user interface (UI) graphical data can be remotely rendered more efficiently at a client during a remote session with a server by utilizing graphical data cached at the client to prevent re-sending data to the client that was sent in previous payloads. By using cache memory to remember recurring frames of similar UI data and delta encoding to correct areas that are not similar, encoded payload sizes are greatly reduced. When a cached frame is closer to a new frame than the current frame, the cached frame is selected and delta commands are derived from differences between the selected cache frame and the new frame. The differences between the cached frame and the new frame are then encoded as delta commands and the cache commands and delta commands are sent to the client which receives and applies the commands to render the new frame.

Optimized key frame caching for remote interface rendering

Raster segment commands are generated by a delta encoder and are encoded for raster segments detected to have moved in the current frame with respect to a location of the raster segment in a previous frame. Raster segment motion commands are stored in a queue and, when written, copy a raster segment into the current frame from a previous payload or cache. When written from the queue to a payload, raster segment motion commands are applied to a copy of the current frame maintained at the server using a copy of the previous payload sent. When copying a raster segment from a raster segment cache, a raster segment cache command retrieves identified raster segment data from the raster segment cache and copies the raster segment into a particular location in the current frame at the client. The raster segment commands are sent to a client via a payload to copy a raster segment from a previous payload into the current frame at the client or from a raster segment cache to the current frame.

Remote user interface raster segment motion detection and encoding

The present invention provides for transferring user interface information from a host computing device to an electronic device that typically has limited resources. A request to display a user interface on an electronic device is received. Further, information about the electronic device is received for determining the capabilities of the electronic device. Based on the determination, a video codec is utilized for dynamically generating a video data stream that includes user interface information corresponding to at least a portion of the user interface. The video data stream is then transmitted to the electronic device for rendering the user interface on a display of the electronic device.

Dynamically generating video streams for user interfaces based on device capabilities

Frames of user interface graphical data can be remotely rendered at a client during a remote session with a server by providing graphical data commands to the client. The commands include motion commands derived from objects that change position between a current frame and a new frame and delta commands derived from differences between the frames. The delta commands can be generated from a frame update after applying motion commands or without applying motion commands. A server identifies moving objects having a first position in the current frame and a second position in the new frame, generates motion hints for the moving objects, and reduces the motion hints based on collision detection, motion verification and other factors. Motion commands are generated for the reduced set of motion hints and applied to a copy of the current frame at the server. Differences between the modified current frame and the new frame are then encoded as delta commands. The server then sends the motion commands and delta commands to the client. The client receives and applies the commands to the current frame to render the new frame.

Growing community of inventors

Redmond, WA, United States of America

Brian C McDowell