Valve Corporation

Alex Vlachos

Aaron Leiby

Kenneth Barnett

Described herein are techniques for adjusting a prediction level and a throttle level, as frames are being rendered on a head-mounted display (HMD), based on an application's rendering performance. The prediction level is increased if a number of late frames, out of a past N rendered frames of (N being any suitable number), meets or exceeds a threshold number of late frames, which causes a compositor of the HMD to predict pose data of the HMD farther out into the future. The throttle level can be increased independently from, or in synchronization with, the increase in the prediction level to causes the compositor to throttle the frame rate of the application (e.g., to a fraction of the refresh rate of the HMD). The prediction level (and the throttle level, if at the same level) can be decreased if a particular number of consecutively-rendered frames finish rendering early.

Prediction and throttling adjustments based on application rendering performance

Systems and methods for implementing radial density masking graphics rendering for use in applications such as head mounted displays ('HMDs') are described. Exemplary algorithms are disclosed, according to which image resolution varies within an image depending on the distance of a particular point on the image from one or more fixation points. Reconstruction algorithms according to certain embodiments include three stages: (1) hole filling; (2) cross-cell blending; and (3) Gaussian blur.

Radial density masking systems and methods

A panel mask(s) rendered on a display panel(s) of a head-mounted display (HMD) may be dynamically adjusted (increased and decreased) in size in order to hide unwanted visual artifacts from view, as needed. For example, if frames are being rendered on the display panel of the HMD using re-projection, a size value associated with at least a portion of the panel mask can be adjusted based on rotation of the HMD to increase or decrease a size of at least the portion of the panel mask from a current size to an adjusted size, and the panel mask can be rendered with at least the portion of the panel mask rendered at the adjusted size to hide the unwanted visual artifacts. The size of the portion of the panel mask can subsequently decrease, over a period of time, if re-projection ceases and/or if head rotation ceases or slows down.

Dynamic panel masking

Described herein are motion smoothing techniques for a display system to account for motion of moving or animating objects in a way that mitigates judder. For example, first pixel data and second pixel data associated with two previously-rendered frames may be provided to a graphics processing unit (GPU) as input. The video encoder of the GPU can process the input pixel data to generate an array of motion vectors which is used to modify third pixel data of a re-projected frame. The modified third pixel data for the re-projected frame is 'motion-smoothed' for rendering on a display, such as a head-mounted display (HMD), in a manner that mitigates judder of moving or animating objects.

Motion smoothing for re-projected frames

Systems and methods for implementing hidden mesh (or stencil mesh) graphics rendering techniques for use in applications such as head mounted displays ('HMDs') are described. Exemplary systems and algorithms are disclosed for masking or eliminating pixels in an image from the list of pixels to be rendered, based on the observation that a significant number of pixels in the periphery of HMD images cannot be seen, due to the specific details of the optical and display/electronics performance of each particular implementation.

Growing community of inventors

Bellevue, WA, United States of America

Alex Vlachos