Microsoft Technology Licensing, LLC

Christian Voss-Wolff

Dag Birger Frommhold

Florian Bagar

Ashraf A Michail

Alex Christopher Turner

Marc Schirski

Thomas Markus Keinz

Matthias Felix Reeh

Techniques for improving remote rendering and reprojection are disclosed herein. A color image is generated, where this color image includes overlapping content regions. A depth buffer is generated for the color image and includes depth values for the pixels in the color image. The depth buffer includes both essential and non-essential depth discontinuities. While preserving the essential depth discontinuities, the non-essential depth discontinuities are eliminated from the depth buffer. New non-essential discontinuities are prevented from being included in the final version of the depth buffer. The color image is encoded into a color image video stream, and the modified depth buffer is encoded into a depth buffer stream. The color image video stream and the depth buffer stream are then transmitted to a remotely located HMD. The HMD then reprojects the color image based on the depth values in the depth buffer.

Depth buffer dilation for remote rendering

A display pose of a client computing device is computed in a hybrid manner using both the client computing device and a server. Using the client computing device, an initial pose of the client computing device at an initial time is computed. A prospective pose at a target time is predicted using the client computing device. The prospective pose is sent along with auxiliary data from the client computing device to the server and a display pose of the client computing device at a display time is computed. The display pose is computed using the prospective pose and the auxiliary data.

Pose prediction for remote rendering

In various examples there is a method performed by a Head Mounted Display, HMD, comprising a high field rate display configured to display fields of rendered frames at a field rate. The method comprises receiving a stream of the rendered frames for display on the high field rate display, the stream of rendered frames having a frame rate. The process applies an early stage reprojection to the rendered frames of the stream of rendered frames at a rate which is lower than the field rate. The process applies a late stage reprojection to fields of the rendered frames at the field rate, wherein the early stage reprojection uses more computational resources than the late reprojection.

Reprojection for high field rate displays

Sending streamed data packets from a producer to a consumer. A method includes, at a first entity, sending consumable data packets from the first entity to a second entity at a first consumable packet rate. The method further includes receiving a first phase delta from the second entity, wherein the first phase delta is computed from transmission jitter, computed from timing information in the consumable data packets. The method further includes sending from the first entity consumable data packets at a second consumable packet rate, the second consumable packet rate being dependent on the first phase delta.

Latency reduction in remote rendering with adaptive phase shifting

Example image reprojection methods are disclosed. In one example, a depth buffer of depth values corresponding to an image produced based on a predicted pose is divided into a plurality of tiles. For each tile of the plurality of tiles, a planar deviation error value that estimates a geometric complexity of the tile and penalizes the tile in proportion to an extent to which a geometry of the tile deviates from a plane is calculated. A tessellated mesh of the plurality of tiles is produced based on the planar deviation error values calculated for the plurality of tiles. An updated pose is received. The tessellated mesh is rendered based on the updated pose to output a reprojected image.

Growing community of inventors

Sonnenbuhl, Germany

Christian Voss-Wolff