Facebook Technologies, LLC

Meta Platforms Technologies, LLC

Larry Seiler

Warren Andrew Hunt

Alexander Nankervis

John Adrian Arthur Johnston

William Thomas Blank

Edward Buckley

Kevin James MacKenzie

Michael Yee

Thomas Scott Murdison

Jeremy Freeman

Alexander Goettker

Alex Nankervis

John Adrian Arthur Johnson

In one embodiment, a method includes identifying a portion of an object representation in a virtual scene to be displayed by a set of pixels of a display, accessing a rendering pattern associated with the set of pixels, wherein the rendering pattern is associated with the set of pixels based on a position of the set of pixels relative to a foveal region, accessing a texture array associated with the portion of the object representation, determining, according to the rendering pattern, first pixel values for a first subset of the set of pixels by sampling the texture array, determining, according to the rendering pattern, second pixel values for a second subset of the set of pixels using the first pixel values, and sending the first pixel values and the second pixel values to the display.

Pixel replication and interpolation for foveated rendering in a display engine

In one embodiment, a computing system may store, by first buffer blocks, texels organized into a texel array including a number of N×N texel sub-arrays. Each texel may fall within a corresponding N×N texel sub-array and may be associated with a two-dimensional sub-array coordinate indicating a position of that texel within the corresponding N×N texel sub-array. Each first buffer block of may be assigned a particular two-dimensional sub-array coordinate and stores a texel subset having the particular two-dimensional sub-array coordinate. The system may receive, by filter blocks, texels from the first buffer blocks. Each filter block may receive a texel from each first buffer block to form a corresponding N×N texel sub-array. The system may perform, by filter blocks, sampling operations parallelly on their respective N×N texel sub-arrays.

Parallel texture sampling

In one embodiment, a computing system may determine a focus point of a viewer based on received sensor data. The system may determine, for a current frame, a first viewing region encompassing a focus point of the viewer and a second view region excluding the first viewing region. The system may determine, for the current frame, color values for the first viewing region using respective first sampling resolutions, and color values for the second viewing region using respective second sampling resolutions. At least one second sampling resolution may be lower than a corresponding first sampling resolution associated with a same color channel. At least two of the second sampling resolutions for the color channels of the second viewing region may be different from each other. The system may output the color values for the first viewing region and the second viewing region of the current frame for display.

Systems and methods for foveated rendering

In an embodiment, a headset display device includes a central processor and multiple projector integrated circuits for eyes of a wearer of the headset display device. Each eye of the wear is associated with at least three projector integrated circuits. Each of the three projector integrated circuits is communicatively coupled to the central processor. Each projector integrated circuit includes a first integrated circuit including a light emitter array having monochrome light emitters of a single color, and a second integrated circuit coupled to the first integrated circuit. The second integrated circuit includes a graphics processor configured to generate transformed image data. The graphics processor is configured to provide the transformed image data to the first integrated circuit. The first integrated circuit is configured to output the transformed image data using the light emitter array.

Multi-projector display architecture

In one embodiment, a method includes accessing display timing information associated with a display, and determining, based on the display timing information, one or more rendering requirements for a plurality of portions of the display. For each portion of the display, the method may determine a current viewpoint of a user, render, based on the current viewpoint of the user, pixel values for that portion of the display in accordance with the associated rendering requirement, and send the rendered pixel values to be output by the display. The display timing information may include a waveform associated with an oscillation motion of a scanning assembly of the display, and/or a requested size for each portion of the plurality of portions of the display. The rendering requirements may include a timing interval for rendering and sending the pixel values for each of the plurality of portions of the display.

Beam-racing pixel generation in a display engine

In one embodiment, a computing system may access an image to be displayed by a display. The system may determine one or more first characteristics associated with a content of the image. The one or more first characteristics may include a contrast level of the content of the image with respect to a background of the image. The system may determine a first display persistence time period for the display to display the image based on the one or more first characteristics associated with the content of the image. The system may configure the display to display the image using the first display persistence time period.

Display artifact reduction

In one embodiment, a computing system may access surfaces and texel data of an artificial reality scene. The surfaces may be generated based on a first viewing position of a viewer. The system may determine tiles on a display to test for a visibility of the surfaces from a second viewing position. The tiles may include first tiles that need more computational resources and second tiles that need less computational resources. The system may determine a tile order which interleaves the first and second tiles. The system may generate rays based on the tile order. The system may determine the visibility of the surfaces from the second viewing position based on the ray-surface intersections. The system may generate color values of a subframe based on the surface visibility and the texel data. The system may provide the color values to the display.

Display engine for post-rendering processing

In one embodiment, a method includes receiving instructions to render a snapshot of a scene for a video, where the snapshot is to be displayed using a sequence of N frames, computing a mipmap-level determining factor for a texture appearing in the scene based on a scale of the texture on a pixel grid, selecting a mipmap level of the texture for each of the N frames based on the mipmap-level determining factor, where the mipmap levels selected for the N frames are non-uniform and temporally approximate the mipmap-level determining factor, rendering each of the N frames by sampling the mipmap level of the texture selected for that frame, and displaying the rendered N frames sequentially to represent the snapshot of the scene.

Temporal approximation of trilinear filtering

In one embodiment, a method includes, by a computing system, accessing a first and second texture associated with an output position, determining a color-blending operation, determining a first color and a first transparency level based on the first texture, determining a second color and a second transparency level based on the second texture, and identifying a color-blending optimization based on the color-blending operation and a comparison of the colors and transparency levels. The method includes determining an output color and an output transparency level by performing the color-blending operation using the colors and transparency levels. The output color is determined by copying the first or second color or the output transparency level is determined by copying the first or second transparency level without additional calculation. The method includes providing the output color and the output transparency level for display at the output position.

Interpolation optimizations for a display engine for post-rendering processing

In one embodiment, a method includes accessing one or more surfaces of an artificial reality scene. The one or more surfaces are generated based on one or more images of the artificial reality scene rendered at a first rate and based on a first viewing position. The method includes generating subframes at a second rate higher than the first rate. Each of the subframes is generated by determining a second viewing position, determining a visibility of the one or more surfaces of the artificial reality scene from the second viewing position, generating, based on the determined visibility of the one or more surfaces, color values of the subframe corresponding to output positions of a display, and providing the color values of the subframe for display.

In one embodiment, a method includes determining a first time interval for rendering pixel values for a first portion of a display, where the first portion of the display is one of a plurality of portions of the display, accessing a representation of virtual objects in a virtual scene, and determining that the first time interval is insufficient for rendering pixel values for the first portion of the display based on the representation of virtual objects. The method further includes, for each of the portions of the display, determining a current viewpoint of a user, rendering, based on the current viewpoint of the user, pixel values for that portion of the display, and sending the rendered pixel values to be output by the display, where the rendered pixel values for the first portion of the display are rendered based on a simplified representation of virtual objects in the virtual scene.

Beam-racing fallbacks in a display engine

In one embodiment, a computing system may access an image to be displayed by a display. The system may determine one or more characteristics associated with a content of the image. The one or more characteristics may include a spatial frequency of the content in a spatial frequency domain. The system may determine a display persistence time period for the display to display the image based on the one or more characteristics associated with the content of the image. The system may configure the display to display the image using the display persistence time period.

In one embodiment, a method includes, generating rays for casting into an artificial reality scene that includes one or more surfaces to determine whether the one or more surfaces are visible from a viewpoint. An origin and a trajectory of each ray are based on the viewpoint. The method includes applying a geometric transformation to the rays to modify their respective trajectory into the artificial reality scene. The geometric transformation is based on one or more distortion characteristics of a display system. The method includes determining, based on the modified trajectories of the rays, points of intersection of rays with the one or more surfaces in the artificial reality scene. The method includes providing, for display by the display system, color values generated based on the determined points of intersection.

In one embodiment, a computing system may determine, for a pixel, a sampling location within a texture that comprises a plurality of texels, and based on the sampling location, select a set of texels in the plurality of texels. Each texel in the set of texels may encode a distance field. An interpolated distance field associated with the sampling location may be computed based on the distance fields associated with the set of texels. The interpolated distance field may represent a relative distance between the sampling location and a texture edge. According to a blending proportion based on the interpolated distance field, the computing system may determine a blended color for the sampling location by blending, a first and second color associated with a first side of the edge and a second side of the edge, and output a color for the pixel based on the blended color.

Anti-aliasing for distance field graphics rendering

In one embodiment, a system may determine a sampling location within a texture with each texel encoding first and second distance fields and first and second color indices. The system may select, based on the sampling location, a set of texels to use to determine a color for the sampling location. The system may compute first and second interpolated distance fields based on, respectively, the first and second distance fields of the set of texels. The system may select, based on the first interpolated distance field, a subset of the set of texels, and select a texel from the subset of texels based on a distance between the texel and the sampling location. The system may select, based on the second interpolated distance filed, a color index from the first and second color indices of the selected texel and use it to determine the color for the sampling location.

Growing community of inventors

Redmond, WA, United States of America

Larry Seiler