Microsoft Technology Licensing, LLC

Eduardo Alberto Cuervo Laffaye

Alastair Wolman

David Chiyuan Chu

Johannes Peter Kopf

Sergey Grizan

Kyungmin Lee

Yury Degtyarev

Stefan Saroiu

Bernhard Reinert

Paramvir Bahl

Ranveer Chandra

Madanlal S Musuvathi

Aruna Balasubramanian

Brian J Smith

Brian Smith

A server device and method are provided for use in predictive server-side rendering of scenes based on client-side user input. The server device may include a processor and a storage device holding instructions for an application program executable by the processor to receive, at the application program, a current navigation input in a stream of navigation inputs from a client device over a network, calculate a predicted future navigation input based on the current navigation input and a current application state of the application program, render a future scene based on the predicted future navigation input to a rendering surface, and send the rendering surface to the client device over the network.

Predictive server-side rendering of scenes

A head mounted display device including a processor configured to compute a rendered rendering surface of a predicted scene having a predicted user viewpoint, the predicted user viewpoint being a prediction of a viewpoint that a user will have at a point in time that was predicted for the user of the head mounted display device prior to the point in time, receive, from the user input device, a subsequent user navigation input near the point in time in the stream of user input, determine an actual user viewpoint based on the subsequent user navigation input, determine a user viewpoint misprediction based on the predicted user viewpoint and the actual user viewpoint, and reconstruct a viewport for the actual user viewpoint from the rendered rendering surface.

Reconstructing viewport upon user viewpoint misprediction

The present describes low latency streaming using temporal frame transformation. An execution component in an edge server executes a first instance of an application. A server interface component receives, from a remote server, a resolution delta frame indicating differences between a high resolution first frame and a low resolution first frame of a second instance of the application or, alternatively, receives the high resolution first frame. A video manipulation component generates a motion delta frame by identifying differences between a low resolution first frame and a low resolution second frame of the first instance of the application. The video manipulation component generates a high resolution transformed frame by applying the resolution delta frame and the motion delta frame to the low resolution second frame.

Low latency application streaming using temporal frame transformation

A first user input is received when a client program executed by a client computing device is in a first state. The first user input is sent to a server computing device to render a view of a virtual scene. A state change from the first state in the client program due to a second user input or a program event is identified. One or more gaps in a server-rendered current view due to the state change are determined. A rendering of the one or more gaps is selected from among the server-rendered current view, a server-rendered predicted view and one or more prior-rendered views. A current view is rendered using a simplified model of the virtual scene by rendering the one or more gaps from the selected rendering. The current rendered view is visually presented via a display of the client computing device.

View rendering from multiple server-side renderings

Various technologies described herein pertain to performing collaborative rendering. A GPU of a mobile device can generate a mobile-rendered video stream based on a first instance of an application executed on the mobile device. A GPU of a server can generate one or more server-rendered video streams based on instance(s) of the application executed on the server. Based on the one or more server-rendered video streams, the server can generate a compressed server-manipulated video stream. The mobile device can further combine the mobile-rendered video stream and the compressed server-manipulated video stream to form a collaborative video stream, and a display screen of the mobile device can be caused to display the collaborative video stream. The mobile-rendered video stream can have a first level of a quality attribute and the collaborative video stream can have a second level of the quality attribute greater than the first level of the quality attribute.

Server GPU assistance for mobile GPU applications

A method described herein includes an act of, at a mobile computing device, receiving an indication that a portion of code of a program executing on the mobile computing device is to be offloaded to a second computing device for execution on the second computing device, wherein the indication is based at least in part upon an estimated energy savings of the mobile computing device by offloading the portion of the code for execution on the second computing device. The method also includes an act of transmitting data to the second computing device that causes the second computing device to execute the portion of the code.

Energy-aware code offload for mobile devices

A client device and method are provided for reconstructing a viewport from a rendered rendering surface of a predicted user viewpoint in order to reduce user perceived latency of the network. The client device may execute instructions to: receive, from a server device over a network, a rendered rendering surface of a predicted scene having a predicted user viewpoint, receive, from the user input device, a subsequent user navigation input in the stream of user input, determine an actual user viewpoint based on the subsequent user navigation input, determine a user viewpoint misprediction based on the predicted user viewpoint and the actual user viewpoint, and reconstruct a viewport for the actual user viewpoint from the rendered rendering surface.

A client device and method are provided for use in synthesizing a second eye viewport using interleaving in order to reduce bandwidth costs. The client device may comprise a user input device that receives a stream of user input, a stereoscopic display device, a processor, and a storage device holding instructions for a client application program, executable by the processor to obtain, at a view interpolation module of the client application program, a current rendered rendering surface representing a current view of a scene for a first eye of a user and a previously rendered rendering surface representing a past view of the scene for a second eye of the user, synthesize and display a current second eye viewport representing a current view of the scene for the second eye of the user based on the current rendered rendering surface and the previously rendered rendering surface.

Growing community of inventors

Bellevue, WA, United States of America

Eduardo Alberto Cuervo Laffaye