Magic Leap, Inc.

Microsoft Technology Licensing, LLC

Tower Semiconductor Ltd.

Microsoft International Holdings B.v.

David Cohen

Giora Yahav

Erez Tadmor

Shlomo Felzenshtein

Sagi Katz

Asaf Pellman

Bradley Vincent Stuart

Daniel Farmer

Eli Larry

Assaf Pellman

Elad Joseph

Amit Shacham

Eyal Preter

Robert D TeKolste

Assaf Lahav

Shai Mazor

Amir Nevet

Eitan Shmuel Bar-On

Ron Nisim Ferens

Shahram Izadi

Scott Charles McEldowney

John Allen Tardif

Robert Dale Tekolste

Amos Fenigstein

John Clavin

Clifford I Drowley

Vladimir Korobov

Erez Sarig

Koon Keong Shee

Avishai Adler

Raz Reshef

Guy Gilboa

Evyatar Bluzer

Shay Alfassi

David Mandelboum

Paul Lacey

Amir Wachs

Ephie Koltin

Shai Kfir

Dani Daniel

Anatoly Sergienko

Hai Reznik

Doron Amihood

Yair Sharf

Naor Inbar

Assaf Pelman

Margalit Ilovich Ayelet

Dalia Oud

Shlomo Felzeshtein

An eye tracking system can include eye-tracking camera(s) configured to obtain images of the eye at different exposure times or different frame rates. For example, longer exposure images of the eye taken at a longer exposure time can show iris or pupil features, and shorter exposure, glint images can show peaks of glints reflected from the eye. The shorter exposure glint images may be taken at a higher frame rate than the longer exposure images for more accurate gaze prediction. The shorter exposure glint images can be analyzed to provide glint locations to subpixel accuracy. The longer exposure images can be analyzed for pupil center and/or center of rotation. The eye tracking system can predict gaze direction, which can be used for foveated rendering by a wearable display system. In some instances, the eye-tracking system may estimate the location of a partially or totally occluded glint.

Systems and techniques for estimating eye pose

Examples of an imaging system for use with a head mounted display (HMD) are disclosed. The imaging system can include a forward-facing imaging camera and a surface of a display of the HMD can include an off-axis diffractive optical element (DOE) or hot mirror configured to reflect light to the imaging camera. The DOE or hot mirror can be segmented, for example, with different segments having different angles or different optical power. The imaging system can be used for eye tracking, biometric identification, multiscopic reconstruction of the three-dimensional shape of the eye, etc. Methods for manufacturing angularly segmented optical elements are also provided. The methods can include injection molding.

Angularly segmented hot mirror for eye tracking

A display system can include a head-mounted display configured to project light to an eye of a user to display virtual image content at different amounts of divergence and collimation. The display system can include an inward-facing imaging system possibly comprising a plurality of cameras that image the user's eye and glints for thereon and processing electronics that are in communication with the inward-facing imaging system and that are configured to obtain an estimate of a center of rotation of the user's eye using cornea data derived from the glint images. The display system may render virtual image content with a render camera positioned at the determined position of the center of rotation of said eye.

Eye center of rotation determination with one or more eye tracking cameras

Techniques are disclosed for operating a time-of-flight (TOF) sensor. The TOF may be operated in a low power mode by repeatedly performing a low power mode sequence, which may include performing a depth frame by emitting light pulses, detecting reflected light pulses, and computing a depth map based on the detected reflected light pulses. Performing the low power mode sequence may also include performing an amplitude frame at least one time by emitting a light pulse, detecting a reflected light pulse, and computing an amplitude map based on the detected reflected light pulse. In response to determining that an activation condition is satisfied, the TOF may be switched to operate in a high accuracy mode by repeatedly performing a high accuracy mode sequence, which may include performing the depth frame multiple times.

Power-efficient hand tracking with time-of-flight sensor

An eye tracking system can include an eye-tracking camera configured to obtain images of the eye at different exposure times or different frame rates. For example, longer exposure images of the eye taken at a longer exposure time can show iris or pupil features, and shorter exposure, glint images can show peaks of glints reflected from the cornea. The shorter exposure glint images may be taken at a higher frame rate (than the longer exposure images) for accurate gaze prediction. The shorter exposure glint images can be analyzed to provide glint locations to subpixel accuracy. The longer exposure images can be analyzed for pupil center or center of rotation. The eye tracking system can predict future gaze direction, which can be used for foveated rendering by a wearable display system. In some instances, the eye-tracking system may estimate the location of a partially or totally occluded glint.

A time of flight based depth detection system is disclosed that includes a projector configured to sequentially emit multiple complementary illumination patterns. A sensor of the depth detection system is configured to capture the light from the illumination patterns reflecting off objects within the sensor's field of view. The data captured by the sensor can be used to filter out erroneous readings caused by light reflecting off multiple surfaces prior to returning to the sensor.

Projector with spatial light modulation

An imaging system includes an illumination unit and a sensor unit disposed on a printed circuit board. The illumination unit includes a diode laser source inside an illumination housing. The sensor unit includes an image sensor having a pixel array and a lens barrel mounted on the image sensor with an adhesive, and an optical fiber coupled between the illumination housing and image sensor. The optical fiber is configured to collect a portion of light from the interior of the illumination housing that is emitted by the diode laser source and direct the portion of light to a corner of the pixel array of the image sensor that is located outside the lens barrel.

Global shutter pixel circuit and method for computer vision applications

An eye tracking system can include an eye-tracking camera configured to obtain images of the eye at different exposure times or different frame rates. For example, longer exposure images of the eye taken at a longer exposure time can show iris or pupil features, and shorter exposure, glint images can show peaks of glints reflected from the cornea. The shorter exposure glint images may be taken at a higher frame rate (than the longer exposure images) for accurate gaze prediction. The shorter exposure glint images can be analyzed to provide glint locations to subpixel accuracy. The longer exposure images can be analyzed for pupil center or center of rotation. The eye tracking system can predict future gaze direction, which can be used for foveated rendering by a wearable display system.

Eye-tracking using images having different exposure times

A method for determining a distance to a target object includes transmitting light pulses to illuminate the target object and sensing, in a first region of a light-sensitive pixel array, light provided from an optical feedback device that receives a portion of the transmitted light pulses. The feedback optical device includes a preset reference depth. The method includes calibrating time-of-flight (TOF) depth measurement reference information based on the sensed light in the first region of the pixel array. The method further includes sensing, in a second region of the light-sensitive pixel array, light reflected from the target object from the transmitted light pulses. The distance of the target object is determined based on the sensed reflected light and the calibrated TOF measurement reference information.

Real time calibration for time-of-flight depth measurement

An image sensor device includes a plurality of pixel cells arranged in a pixel array, a control circuit for controlling an exposure phase and a sampling phase of the image sensor device. Each of the plurality of pixel cells includes a photodiode, a storage diode, and a floating diffusion region. The control circuit is configured to activate the photodiode in a plurality of time windows to sense light reflected from a target as a result of a corresponding plurality of emitted light pulses, with a pre-determined delay time between each time window and a corresponding emitted light pulse. The photodiode can be activated using a plurality of bias voltage pulses or a plurality of global shutter signal pulses.

Growing community of inventors

Nesher, Israel

David Cohen