Facebook Technologies, LLC

Microsoft Technology Licensing, LLC

Meta Platforms Technologies, LLC

Alexander Nankervis

Warren Andrew Hunt

Larry Seiler

Anton S Kaplanyan

Michael Mara

John Adrian Arthur Johnston

Mark Stefan Grossman

Jason Matthew Gould

Jeremy Freeman

Charles Neill Boyd

Blake Douglas Pelton

Matthew L Bronder

Jack Elliott

Matthew David Sandy

Andrew Zicheng Yeung

Jack A Elliott

John Adrian Arthur Johnson

In one embodiment, a computing system may access surfaces and texel data of an artificial reality scene. The surfaces may be generated based on a first viewing position of a viewer. The system may determine tiles on a display to test for a visibility of the surfaces from a second viewing position. The tiles may include first tiles that need more computational resources and second tiles that need less computational resources. The system may determine a tile order which interleaves the first and second tiles. The system may generate rays based on the tile order. The system may determine the visibility of the surfaces from the second viewing position based on the ray-surface intersections. The system may generate color values of a subframe based on the surface visibility and the texel data. The system may provide the color values to the display.

Display engine for post-rendering processing

In one embodiment, a method includes accessing one or more surfaces of an artificial reality scene. The one or more surfaces are generated based on one or more images of the artificial reality scene rendered at a first rate and based on a first viewing position. The method includes generating subframes at a second rate higher than the first rate. Each of the subframes is generated by determining a second viewing position, determining a visibility of the one or more surfaces of the artificial reality scene from the second viewing position, generating, based on the determined visibility of the one or more surfaces, color values of the subframe corresponding to output positions of a display, and providing the color values of the subframe for display.

In one embodiment, a method includes determining a first time interval for rendering pixel values for a first portion of a display, where the first portion of the display is one of a plurality of portions of the display, accessing a representation of virtual objects in a virtual scene, and determining that the first time interval is insufficient for rendering pixel values for the first portion of the display based on the representation of virtual objects. The method further includes, for each of the portions of the display, determining a current viewpoint of a user, rendering, based on the current viewpoint of the user, pixel values for that portion of the display, and sending the rendered pixel values to be output by the display, where the rendered pixel values for the first portion of the display are rendered based on a simplified representation of virtual objects in the virtual scene.

Beam-racing fallbacks in a display engine

In one embodiment, a method includes, generating rays for casting into an artificial reality scene that includes one or more surfaces to determine whether the one or more surfaces are visible from a viewpoint. An origin and a trajectory of each ray are based on the viewpoint. The method includes applying a geometric transformation to the rays to modify their respective trajectory into the artificial reality scene. The geometric transformation is based on one or more distortion characteristics of a display system. The method includes determining, based on the modified trajectories of the rays, points of intersection of rays with the one or more surfaces in the artificial reality scene. The method includes providing, for display by the display system, color values generated based on the determined points of intersection.

In one embodiment, a system may determine a sampling location within a texture with each texel encoding first and second distance fields and first and second color indices. The system may select, based on the sampling location, a set of texels to use to determine a color for the sampling location. The system may compute first and second interpolated distance fields based on, respectively, the first and second distance fields of the set of texels. The system may select, based on the first interpolated distance field, a subset of the set of texels, and select a texel from the subset of texels based on a distance between the texel and the sampling location. The system may select, based on the second interpolated distance filed, a color index from the first and second color indices of the selected texel and use it to determine the color for the sampling location.

Dual distance field color palette

In one embodiment, a method includes accessing a first distortion definition for a first color component and a second distortion definition for a second color component; generating, for a pixel, a first ray for the first color component of the pixel and a second ray for the second component of the pixel; defining trajectories for the first and second rays using the first and second distortion definitions, respectively; casting, from a viewpoint in a 3D space, the first and second rays into the 3D space using the defined trajectories; computing, for the first and second rays, a first intersection location and a second intersection location with one or more objects in the 3D space; and determining a color value for the pixel based on the first and second intersection locations.

Multi-channel ray casting with distortion meshes to address chromatic aberration

In one embodiment, a computing system may determine an orientation in a three-dimensional (3D) space and generate a plurality of coordinates in the 3D space based on the determined orientation. The system may access pre-determined ray trajectory definitions associated with the plurality of coordinates. The system may determine visibility information of one or more objects defined within the 3D space by projecting rays through the plurality of coordinates, wherein trajectories of the rays from the plurality of coordinates are determined based on the pre-determined ray trajectory definitions. The system may then generate an image of the one or more objects based on the determined visibility information of the one or more objects.

Systems and methods for rendering optical distortion effects

In one embodiment, a computing system may determine a first orientation of a viewer in a three-dimensional (3D) space based on first sensor data associated with a first time. The system may render one or more first lines of pixels based on the first orientation of the viewer and display the one or more first lines. The system may determine a second orientation of the viewer in the 3D space based on second sensor data associated with a second time that is subsequent to the first time. The system may render one or more second lines of pixels based on the second orientation of the viewer and display the one or more second lines of pixels. The one or more second lines of pixels associated with the second orientation are displayed concurrently with the one or more first lines of pixels associated with the first orientation.

Systems and methods for reducing rendering latency

In one embodiment, a method for determining the color for a sample location includes using a computing system to determine a sampling location within a texture that comprises a plurality of texels. Each texel may encode a distance field and a color index. The system may select, based on the sampling location, a set of texels in the plurality of texels to use to determine a color for the sampling location. The system may compute an interpolated distance field based on the distance fields of the set of texels. The system may select, based on the interpolated distance field, a subset of the set of texels. The system may select a texel from the subset of texels based on a distance between the texel and the sampling location. The system may then determine the color for the sampling location using the color index of the selected texel.

Distance field color palette

In one embodiment, a method for computing a color value for a sampling pixel region includes using a computing system to determine a sampling pixel region within a texture. The texture is associated with mipmap levels having different resolutions of the texture. The mipmap levels include at least a first mipmap level defined by color texels and a second mipmap level defined by distance-field texels. The system may select one of the mipmap levels based on a size of the sampling pixel region and a size of a texel in the selected mipmap level. The system may then compute a color value for the sampling pixel region using the selected mipmap level.

Mipmap with mixed texture types

In one embodiment, a method for determine visibility may perform intersection tests using block beams, tile beams, and rays. First, a computing system may project a block beam to test for intersection with a first bounding volume (BV) in a bounding volume hierarchy. If the beam fully contains BV, the system may test for more granular intersections with the first BV by projecting smaller tile beams contained within the block beam. Upon determining that the first BV partially intersects a tile beam, the system may project the tile beam against a second BV contained within the first BV. If the tile beam fully contains the second BV, the system may test for intersection using rays contained within the tile beam. The system may project procedurally-generated rays to test whether they intersect with objects contained within the second BV. Information associated with intersections may be used to render a computer-generated scene.

Computer-graphics based on hierarchical ray casting

In one embodiment, a computing system may determine a first orientation in a 3D space based on first sensor data generated at a first time. The system may determine a first visibility of an object in the 3D space by projecting rays based on the first orientation to test for intersection. The system may generate first lines of pixels based on the determined first visibility and output the first lines of pixels for display. The system may determine a second orientation based on second sensor data generated at a second time. The system may determine a second visibility of the object by projected rays based on the second orientation to test for intersection. The system may generate second lines of pixels based on the determined second visibility and output the second lines of pixels for display. The second lines of pixels are displayed concurrently with the first lines of pixels.

In one embodiment, a computer system may determine an orientation in a 3D space based on sensor data generated by a virtual reality device. The system may generate ray footprints in the 3D space based on the determined orientation. For at least one of the ray footprints, the system may identify a corresponding number of subsamples to generate for that ray footprint and generate one or more coordinates in the ray footprint based on the corresponding number of subsamples. The system may determine visibility of one or more objects defined within the 3D space by projecting a ray from each of the one or more coordinates to test for intersection with the one or more objects. The system may generate an image of the one or more objected based on the determined visibility of the one or more objects.

Systems and methods for rendering foveated effects

Methods and devices for rendering graphics in a computer system include a graphical processing unit (GPU) with a flexible, dynamic, application-directed mechanism for varying the rate at which fragment shading is performed for rendering an image to a display. In particular, the described aspects allow different shading rates to be used for different regions of a primitive based on a new, interpolated shading rate parameter. In other words, the described aspects enable the GPU to change shading rates on-the-fly between different fragments of each primitive. Additionally, or independently, the GPU utilizes each respective shading rate parameter to determine how many sample positions to consider to be covered by the computed shaded output, e.g., the fragment color, thereby allowing the color sample to be shared across two or more pixels.

Variable rate shading

In one embodiment, a computing system may receive a focal surface map, which may be specified by an application. The system may determine an orientation in a 3D space based on sensor data generated by a virtual reality device. The system may generate first coordinates in the 3D space based on the determined orientation and generate second coordinates using the first coordinates and the focal surface map. Each of the first coordinates is associated with one of the second coordinates. For each of the first coordinates, the system may determine visibility of one or more objects defined within the 3D space by projecting a ray from the first coordinate through the associated second coordinate to test for intersection with the one or more objects. The system may generate an image of the one or more objected based on the determined visibility of the one or more objects.

Growing community of inventors

Seattle, WA, United States of America

Alexander Nankervis