Facebook Technologies, LLC

Meta Platforms Technologies, LLC

Thomas Scott Murdison

Robert Dale Cavin

Karol Constantine Hatzilias

Kevin James MacKenzie

Martin Henrik Tall

Sharvil Shailesh Talati

Neeraj Choubey

Romain Bachy

Bo Asp Andersen

Andre Golard

Immo Andreas Schuetz

Kelly Ingham

Edward Buckley

Jasmine Soria Sears

Nathan Seigo Matsuda

Charlene Mary Atlas

Ocean Quigley

Larry Seiler

Bo Zhang

Alexander Goettker

Andrew John Ouderkirk

Alexander Jobe Fix

Sebastian Sztuk

Javier San Agustin Lopez

Kirk Erik Burgess

Scott Robert Ramsby

Mohammadhossein Daraeihajitooei

Mohamed Hegazy

Andrew Wyman MacDonald

Dmitri Model

Evan Gander

Jixu Chen

Xu Dong

An example method is provided for applying a virtual lighting effect to virtual content based on ambient light associated with a real-world scene. The example method includes obtaining a measurement of ambient light associated with a scene of a real-world environment. The example method includes applying a virtual lighting effect to virtual content based at least in part on the measurement of the ambient light. And the example method includes causing display of the virtual content with the applied virtual lighting effect on a display of a computing device for displaying augmented reality, wherein the scene of the real-world environment is visible through the display.

Techniques for adaptively rendering virtual content n artificial reality environments based on ambient lighting of a real-world scene

In one embodiment, a computing device may determine a virtual content to be displayed with a scene of a real-world environment. The device may generate an image depicting the virtual content. Using one or more sensors, the device may detect characteristics of the scene of the real-world environment. Based on the image and the characteristics of the scene, the device may determine that a visual enhancement is to be applied to the virtual content depicted in the image to enhance a contrast between the depicted virtual content and the scene. The device may generate a visually-enhanced image depicting the virtual content by applying the visual enhancement to the virtual content depicted in the image. The device may display the visually-enhanced image of the virtual content on a display of the computing device, wherein the scene of the real-world environment is visible through the display.

Adaptive rendering in artificial reality environments

In one embodiment, a computing system may determine, for a current frame, that a current eye position of a viewer with respect to a display is inside and within a first threshold distance to an outer edge of the display. The system may identify, based on the current eye position, pre-determined internal eye positions inside the outer edge and pre-determined external eye positions outside the outer edge, obtain pre-determined arrays of scaling factors associated with the pre-determined internal eye positions, and obtain additional arrays of scaling factors associated with the pre-determined external eye positions. The system may generate a single array of scaling factors based on the pre-determined and additional arrays, adjust pixel values of the current frame based on the single array, and output the current frame with the adjusted pixel values to the display. The arrays and adjusted pixel values may be associated with a particular color channel.

Dynamic uniformity correction for boundary regions

In one embodiment, a computing system may access an image to be displayed by a display. The system may determine one or more first characteristics associated with a content of the image. The one or more first characteristics may include a contrast level of the content of the image with respect to a background of the image. The system may determine a first display persistence time period for the display to display the image based on the one or more first characteristics associated with the content of the image. The system may configure the display to display the image using the first display persistence time period.

Display artifact reduction

In one embodiment, a computing system may determine, a predicted eye position of a viewer corresponding to a future time moment for displaying a frame. The system may generate a first correction map for the frame based on the predicted eye position of the viewer. The system may retrieve one or more second correction maps used for correcting one or more proceeding frames. The system may generate a third correction map based on the first correction map generated based on the predicted eye position of the viewer and the one or more second correction maps used for correcting the one or more proceeding frames. The system may adjust pixel values of the frame based at least on the third correction map. The system may output the frame with the adjusted pixel values to a display.

Dynamic uniformity correction

In one embodiment, a computing system may render an image to be output by a display which sequentially outputs pixel rows of the image at a row-to-row velocity. The system may predict, based on eye-tracking data, an eye-motion velocity within a timeframe for displaying the image. The system may determine a predicted retina projection displacement for each pixel row based on the row-to-row velocity and eye-motion velocity. The system may determine a first correction displacement for each pixel row based on the predicted retina projection displacement. The system may determine a cumulative correction displacement based on the first correction displacement. The system may determine, for each pixel row, a second correction displacement based on the first correction displacement of that pixel row and the cumulative correction displacement. The system may warp each pixel row using the associated second correction displacement and output the warped pixel rows using the display.

Image correction

A method for color-based calibration for eye-tracking includes presenting a calibration image to an eyebox area via a waveguide of a head mounted display (HMD). An estimated pupil center location of an eye is determined with an eye-tracking system. Next, user input is received in response to the calibration image. The user input includes an indication of a perceived color nonuniformity of the calibration image. The estimated pupil center location is then adjusted based on the user input.

Color-based calibration for eye-tracking

In one embodiment, a computing system may determine, for a current frame to be displayed and using an eye tracking system, a current eye position of a viewer. The system may determine a first array of scaling factors based on the determined current eye position of the viewer. The system may retrieve one or more second arrays of scaling factors used for correcting one or more proceeding frames of the current frame. The system may determine a third array of scaling factors based on the first array of scaling factors determined based on the current eye position and the one or more second arrays of scaling factors used for correcting the proceeding frames. The system may adjust pixel values of the current frame based at least on the third array of scaling factors. The system may output the current frame with the adjusted pixel values to a display.

In one embodiment, a computing system may access an image to be displayed by a display. The system may determine one or more characteristics associated with a content of the image. The one or more characteristics may include a spatial frequency of the content in a spatial frequency domain. The system may determine a display persistence time period for the display to display the image based on the one or more characteristics associated with the content of the image. The system may configure the display to display the image using the display persistence time period.

A disclosed apparatus may include a line source configured to produce a line of light and a scanning device configured to scan the line of light across a scanning field in a scanning direction. The scanning field may include a receiving portion configured to receive an eye. The apparatus may also include a reflector positioned within the scanning field. During a primary period of a scan, the line of light may scan the receiving portion in a primary direction. During a secondary period of the scan, the reflector may reflect the line of light such that a reflection of the line of light scans the receiving portion in a secondary direction. The apparatus may also include a photodetector positioned to receive (1) an initial reflection during the primary period, and (2) a subsequent reflection during the secondary period. Various other methods, apparatuses, and computer-readable media are also disclosed.

Apparatuses, systems, and methods for scanning an eye via a folding mirror

The disclosed computer-implemented method may include identifying a region within a user's eye gaze and calculating a ranking for the identified region within the user's eye gaze. The ranking may indicate the user's level of interest in the identified region. The method may then determine how the identified region is to be presented according to the calculated ranking and present the identified region in the determined manner according to the calculated ranking. Various other methods, systems, and computer-readable media are also disclosed.

Eye rank

Systems and methods for virtual interest segmentation may include (1) performing a semantic segmentation of an image of a user's environment, captured by an artificial reality (AR) device being worn by the user, to identify objects within the user's environment, (2) in addition to performing the semantic segmentation, performing an interest segmentation of the image to determine a personal interest that the user may have in a particular object identified via the semantic segmentation, (3) creating virtual content relating to the particular object based on the user's personal interest in the particular object, and (4) displaying the virtual content within a display element of the AR device. Various other methods, systems, and computer-readable media are also disclosed.

Virtual interest segmentation

Systems and methods for enabling gaze-based virtual content control may include (1) displaying an artificial scene with one or more virtual elements to a user wearing a head-mounted display system, (2) identifying the user's eye gaze based on gazing data collected by one or more sensors in the head-mounted display system, (3) determining that the user's eye gaze is focused on a specific virtual element, and (4) in response to determining that the user's eye gaze is focused on the specific virtual element, increasing the specific virtual element's visibility to the user. Various other methods, systems, and computer-readable media are also disclosed.

Gaze-based virtual content control

The disclosed computer-implemented method may include: identifying, using an eye-tracking system, an object within a scene viewed by a user; identifying, within a database of object interaction commands, a subset of commands that apply to the object viewed by the user; and presenting, to the user, the subset of commands that apply to the object. Various other methods, systems, devices, and computer-readable media are also disclosed.

Growing community of inventors

Kirkland, WA, United States of America

Thomas Scott Murdison