Magic Leap, Inc.

Bradley Vincent Stuart

Daniel Farmer

Samuel A Miller

Edwin Joseph Selker

Jeffrey Scott Sommers

Lomesh Agarwal

Yonatan Munk

Sergey Fyodorovich Prokushkin

Lionel Ernest Edwin

Ivan Li Chuen Yeoh

David Cohen

Paul M Greco

Yan Xu

Erik Fonseka

Shiuli Das

Suraj Manjunath Shanbhag

Benjamin Joseph Uscinski

Tiejian Zhang

Ivan Li Yeoh

A display system includes a head-mounted display configured to project light, having different amounts of wavefront divergence, to an eye of a user to display virtual image content appearing to be disposed at different depth planes. The wavefront divergence may be changed in discrete steps, with the change in steps being triggered based upon whether the user is fixating on a particular depth plane. The display system may be calibrated for switching depth planes for a main user. Upon determining that a guest user is utilizing the system, rather than undergoing a full calibration, the display system may be configured to switch depth planes based on a rough determination of the virtual content that the user is looking at. The virtual content may have an associated depth plane and the display system may be configured to switch to the depth plane of that virtual content.

Depth plane selection for multi-depth plane display systems by user categorization

An eye tracking system can include eye-tracking camera(s) configured to obtain images of the eye at different exposure times or different frame rates. For example, longer exposure images of the eye taken at a longer exposure time can show iris or pupil features, and shorter exposure, glint images can show peaks of glints reflected from the eye. The shorter exposure glint images may be taken at a higher frame rate than the longer exposure images for more accurate gaze prediction. The shorter exposure glint images can be analyzed to provide glint locations to subpixel accuracy. The longer exposure images can be analyzed for pupil center and/or center of rotation. The eye tracking system can predict gaze direction, which can be used for foveated rendering by a wearable display system. In some instances, the eye-tracking system may estimate the location of a partially or totally occluded glint.

Systems and techniques for estimating eye pose

Systems and methods for eye tracking latency enhancements. An example head-mounted system obtains a first image of an eye of a user. The first image is provided as input to a machine learning model which has been trained to generate iris and pupil segmentation data given an image of an eye. A second image of the eye is obtained. A set of locations in the second image at which one or more glints are shown is detected based on iris segmentation data generated for the first image. A region of the second image at which the pupil of the eye of the user is shown is identified based on pupil segmentation data generated for the first image. A pose of the eye of the user is determined based on the detected set of glint locations in the second image and the identified region of the second image.

Eye tracking latency enhancements

A display system can include a head-mounted display configured to project light to an eye of a user to display virtual image content at different amounts of divergence and collimation. The display system can include an inward-facing imaging system that images the user's eye and processing electronics that are in communication with the inward-facing imaging system and that are configured to obtain an estimate of a center of rotation of the user's eye. The display system may render virtual image content with a render camera positioned at a determined position relative to the determined position of the center of rotation of said eye.

Eye center of rotation determination, depth plane selection, and render camera positioning in display systems

An eye tracking system can include an eye-tracking camera configured to obtain images of the eye at different exposure times or different frame rates. For example, longer exposure images of the eye taken at a longer exposure time can show iris or pupil features, and shorter exposure, glint images can show peaks of glints reflected from the cornea. The shorter exposure glint images may be taken at a higher frame rate (than the longer exposure images) for accurate gaze prediction. The shorter exposure glint images can be analyzed to provide glint locations to subpixel accuracy. The longer exposure images can be analyzed for pupil center or center of rotation. The eye tracking system can predict future gaze direction, which can be used for foveated rendering by a wearable display system. In some instances, the eye-tracking system may estimate the location of a partially or totally occluded glint.

A display system includes a head-mounted display configured to project light, having different amounts of wavefront divergence, to an eye of a user to display virtual image content appearing to be disposed at different depth planes. The wavefront divergence may be changed in discrete steps, with the change in steps being triggered based upon whether the user is fixating on a particular depth plane. The display system may be calibrated for switching depth planes for a main user. Upon determining that a guest user is utilizing the system, rather than undergoing a full calibration, the display system may be configured to switch depth planes based on a rough determination of the virtual content that the user is looking at. The virtual content has an associated depth plane and the display system may be configured to switch to the depth plane of that virtual content.

A display system can include a head-mounted display configured to project light to an eye of a user to display virtual image content at different amounts of divergence and collimation. The display system can include an inward-facing imaging system images the user's eye and processing electronics that are in communication with the inward-facing imaging system and that are configured to obtain an estimate of a center of rotation of the user's eye. The display system may render virtual image content with a render camera positioned at the determined position of the center of rotation of said eye.

Systems and methods for eye tracking calibration in a wearable system are described. The wearable system can present three-dimensional (3D) virtual content and allow a user to interact with the 3D virtual content using eye gaze. During an eye tracking calibration, the wearable system can validate that a user is indeed looking at a calibration target while the eye tracking data is acquired. The validation may be performed based on data associated with the user's head pose and vestibulo-ocular reflex.

Growing community of inventors

Fort Lauderdale, FL, United States of America

Bradley Vincent Stuart