Microsoft Technology Licensing, LLC

Eduardo Cuervo

David Chiyuan Chu

Eric J Horvitz

Paramvir Bahl

Alastair Wolman

Manikanta Kotaru

Krishna Chintalapudi

Lenin Ravindranath Sivalingam

Monia Ghobadi

Kiryong Ha

Kevin Boos

A server may be configured to receive first pose data from a virtual reality device, generate second pose data, including a pose prediction, based at least on the first pose data, render a map representation based at least on the pose prediction, determine regions of the map representation based on a likelihood of view of each of the regions, perform foveation in each of the regions and generate an encoded frame based on the regions and a network parameter, and send the encoded frame and second pose data to the device. The encoding may associate each of the regions with a quality level and base the foveation on each region's associated quality level. The foveation may use different quantization parameters for each of the regions and/or use rendering foveation for each of regions. Network conditions such as latency, bandwidth, and/or jitter may be considered for bit allocation in the foveation.

Virtual reality with interactive streaming video and likelihood-based foveation

A virtual reality system including a head mounted display (HMD), a server, and a beam steering apparatus utilizes optical data transmission from the server to the HMD. The data rates/bandwidths provided by optical data transmission allow the amount of compression needed for data transfer (and the latency associated with compression) to the HMD to be reduced. The efficient offloading of processing tasks from the HMD to the server, reduces HMD power consumption, and enables the delivery of video having high resolution, framerate, and quality. As the user of the HMD moves, and the pose (position and orientation) of the HMD changes, the pose is provided to the server and beam steering apparatus. Based on the pose, the server renders image frames for transmission to the HMD and the beam steering apparatus directs an optical beam to the HMD to enable transmission of the image frames from the server to the HMD.

Mixed reality offload using free space optics

Virtual reality is provided by a mobile VR device. VR scenes are displayed by serving an application's image requests from a local cache of pre-rendered image frames. As the device user moves, image frames that match a VR scene viewed from a current pose (3D position and 3D orientation of the device translated to the virtual environment) are retrieved from the cache and displayed to the user as a VR scene. The cache may include a static cache including static image frames and a dynamic object cache including dynamic object image frames. The static cache may be indexed by the device current pose. The dynamic object cache may be indexed by the dynamic object's animation stage, orientation, and relative distance from the device user's current pose for a given time or movement-based trigger. Static image frames and dynamic object image frames may be composited, and displayed as a scene.

Stereoscopic virtual reality through caching and image based rendering

The claimed subject matter includes techniques for live migration of a graphics processing unit (GPU) state. An example method includes receiving recorded GPU commands from a relay at a destination GPU. The method also includes replaying the recorded GPU commands at the destination GPU. The method also includes detecting a downtime for the GPU commands. The method further includes establishing a connection between the destination GPU and the client during the detected downtime.

Migration of graphics processing unit (GPU) states

Growing community of inventors

Bellevue, WA, United States of America

Eduardo Cuervo