Facebook Technologies, LLC

Facebook, Inc.

Meta Platforms Technologies, LLC

Javier San Agustin Lopez

Sebastian Sztuk

Martin Henrik Tall

Alexander Jobe Fix

Scott Robert Ramsby

Mohamed Hegazy

Dmitri Model

Kirk Erik Burgess

Mohammadhossein Daraeihajitooei

Andrew Wyman MacDonald

Robert Dale Cavin

Karol Constantine Hatzilias

Steven Paul Lansel

Rasmus Dahl

Robin Sharma

Anders Bo Pedersen

John Paulin Hansen

Andrew John Ouderkirk

Kurt Jenkins

Sheng Ye

Sapna A Shroff

Kun Liu

Thomas Scott Murdison

Neeraj Choubey

Wei Rong

Bo Asp Andersen

Evan Gander

Jixu Chen

Henrik Hegner Tomra Skovsgaard

Jonas Priesum

Sune Alstrup Johansen

Mohammadhossein Daraeihajitooel

A disclosed computer-implemented method may include directing a display device included in a head-mounted display worn by a user to illuminate, via a projection of a line at an illumination time, a portion of a cornea of the user. The method may further include detecting, via an image sensor at a detection time, a portion of the projection of the line reflected by the portion of the cornea of the user and identifying a distortion of the projection of the line reflected by the portion of the cornea of the user. The method may also include determining a shape of the cornea of the user based on the illumination time, the detection time, and the distortion of the projection of the line reflected by the portion of the cornea. Various other methods, systems, apparatuses, and computer-readable media are also disclosed.

Apparatuses, systems, and methods for mapping corneal curvature

Various aspects of the subject technology relate to prediction of eye movements of a user of a head-mountable display device. Predictive foveated display systems and methods, using the predicted eye movements are also disclosed. Predictive variable focus display systems and methods using the predicted eye movements are also disclosed. Predicting eye movements may include predicting a future gaze location and/or predicting a future vergence plane for the user's eyes, based on the current motion of one or both of the user's eyes. The predicted gaze location may be used to pre-render a foveated display image frame with a high-resolution region at the predicted gaze location. The predicted vergence plane may be used to modify an image plane of a display assembly to mitigate or avoid a vergence/accommodation conflict for the user.

Predictive eye tracking systems and methods for foveated rendering for electronic displays

The disclosed eye-tracking systems may include a visible light display having visible light pixels, at least one infrared pixel positioned within bounds of the visible light display, and an infrared sensor positioned and configured to detect infrared light originating from the at least one infrared pixel and reflected from an eye of the user. Related head-mounted display systems and methods for eye tracking are also disclosed.

Systems and methods for using a display as an illumination source for eye tracking

An optical assembly includes an optically transparent substrate and a first lens removably coupled with the optically transparent substrate by one or more adhesive layers. The one or more adhesive layers are positioned between the optically transparent substrate and the first lens. Also disclosed are an eye-tracker and a display device, each of which includes the optical assembly.

Head-mounted display devices with removable prescription lenses

A method for calibrating eye tracking data for a head wearable optical device includes receiving eye tracking data from an imaging device oriented towards a combiner. The method includes determining a gaze direction of an eye of a user based on the eye tracking data. The method includes receiving calibration data from the imaging device that includes an indication of positions of one or more calibration indicators. The method includes determining the gaze direction based at least in part on the calibration data received from the imaging device (or factors adjusted by the calibration data).

Systems and methods for calibrating eye tracking

A hybrid cornea and pupil eye-tracking technique includes receiving iris-backscattered light from an eye and determining a pupil-position of the eye based on the iris-backscattered light. Cornea-reflected light is received from the eye and an eye-tracking position of the eye is determined based on the cornea-reflected light and the pupil-position.

Hybrid cornea and pupil tracking

A virtual reality (VR) system includes a console an imaging device, a head mounted display (HMD) and a user input device. The console includes a virtual reality (VR) engine, a tracking module and an application store. The HMD includes a display element configured to display content to a user wearing the HMD. The virtual reality engine is configured to determine a vector for a virtual object thrown at a target in response to physical input from the user, a gaze vector for the eyes of the user and a virtual object position vector for the target and modify the vector for the virtual object based on at least one of the gaze vector and the virtual object position vector. The virtual reality engine may also be configured to determine a virtual object position vector for a virtual object to be thrown at a target, a gaze vector for the eyes of the user and a virtual object position vector for a virtual rendering of the user's hand and enabling user input to pick up the virtual object based on at least one of the gaze vector and the virtual object position vector of the virtual rendering of the user's hand.

User interaction in head-mounted display with eye tracking

A disclosed computer-implemented method may include directing an illumination source, positioned at a predetermined position relative to an event camera positioned to receive light from the illumination source reflected by a portion of a cornea of a user during an illumination sequence, to illuminate the portion of the cornea of the user in accordance with the illumination sequence. The method may also include detecting, via the event camera, a set of events, each event in the set of events comprising a change in an intensity of light from the illumination source received by a pixel included in the event camera over a sampling period of the event camera. The method may further include determining a shape of the portion of the cornea of the user based on the predetermined position, the illumination sequence, and the set of events. Various other apparatuses, systems, methods, and computer-readable media are also disclosed.

Apparatuses, systems, and methods for mapping a surface of an eye via an event camera

A disclosed apparatus may include a line source configured to produce a line of light and a scanning device configured to scan the line of light across a scanning field in a scanning direction. The scanning field may include a receiving portion configured to receive an eye. The apparatus may also include a reflector positioned within the scanning field. During a primary period of a scan, the line of light may scan the receiving portion in a primary direction. During a secondary period of the scan, the reflector may reflect the line of light such that a reflection of the line of light scans the receiving portion in a secondary direction. The apparatus may also include a photodetector positioned to receive (1) an initial reflection during the primary period, and (2) a subsequent reflection during the secondary period. Various other methods, apparatuses, and computer-readable media are also disclosed.

Apparatuses, systems, and methods for scanning an eye via a folding mirror

A system is describes that includes a head mounted display (HMD) and a portable docking station configured to receive the HMD for calibration of one or more components of the HMD. The portable docking station includes at least one calibration target, e.g., a checkerboard pattern and/or a convex reflector. Techniques of this disclosure include calibrating an image capture device of the HMD based on one or more images of the calibration target captured by the image capture device when the HMD is placed in the portable docking station. The disclosed techniques may be applied to calibrate multiple different components of the HMD, including image capture devices such as eye-tracking cameras and inside-out cameras, displays, illuminators, sensors, and the like. In some examples, a rechargeable battery of the HMD may be charged when the HMD is placed in the portable docking station.

Head mounted display calibration using portable docking station with calibration target

A method of notifying a user about attention from another user in an augmented reality/virtual reality (AR/VR) system is provided. The method includes displaying a first image on a first electronic display to a first user of the AR/VR system and, upon detecting a gaze of the first user at a second user of the AR/VR system or the second user's avatar in the first image, notifying the second user about the gaze of the first user by displaying a second image on a second electronic display to the second user, the second image having a visual cue of the gaze of the first user.

Visual indicators of user attention in AR/VR environment

A method of notifying a user about attention from another user in an augmented reality/virtual reality (AR/VR) system is provided. The method includes displaying a first image on a first electronic display to a first user of the AR/VR system and, upon detecting a gaze of the first user at a second user of the AR/VR system or the second user's avatar in the first image, notifying the second user about the gaze of the first user by changing a parameter of an audio communication channel from the first user to the second user, so as to make the first user more audible to the second user.

Growing community of inventors

Menlo Park, CA, United States of America

Javier San Agustin Lopez