Facebook Technologies, LLC

Meta Platforms Technologies, LLC

Qing Chao

Michael Anthony Hall

Xinqiao Liu

Zhaoming Zhu

Lu Lu

Byron Taylor

Robert Dale Cavin

Tin Komljenovic

Zihe Gao

Hao Yu

Robin Sharma

Scott Charles McEldowney

Qi Zhang

Junren Wang

Mohammadhossein Daraeihajitooei

Ziyun Li

Sascha Hallstein

Elias Daniel Guestrin

Guillaume Lestoquoy

Raffaele Capoccia

Seth Michael Hirsh

An eye-tracking system includes a substrate transparent to visible light and configured to be placed in a field of view of an eye of a user, a plurality of waveguides on the substrate, a light source optically coupled to the plurality of waveguides, and a plurality of polarization volume holograms (PVHs) in the field of view of the eye of the user. Each PVH of the plurality of PVHs is optically coupled to a respective waveguide of the plurality of waveguides and is configured to couple a respective light beam out of the respective waveguide towards the eye of the user.

Liquid crystal polarization hologram (LCPH) based eye tracking for AR/VR

Imaging systems, cameras, and image sensors of this disclosure include imaging pixels that include subpixels. Diffractive optical elements such as a metasurface lens layers or a liquid crystal polarization hologram (LCPH) are configured to focus image light to the subpixels of the imaging pixels.

Sensing with liquid crystal polarization holograms and metasurface

An eyebox region is illuminated with a fringe illumination pattern. An event sensor is configured to generate event-signals. Eye motion is determined from the event-signals. Eye-features are extracted from data generated by the event sensors and a predicted gaze vector is generated from the eye-features.

Event sensing and tracking

A depth camera assembly (DCA) for depth sensing of a local area. The DCA includes a transmitter, a receiver, and a controller. The transmitter illuminates a local area with outgoing light in accordance with emission instructions. The transmitter includes a fine steering element and a coarse steering element. The fine steering element deflects one or more optical beams at a first deflection angle to generate one or more first order deflected scanning beams. The coarse steering element deflects the one or more first order deflected scanning beams at a second deflection angle to generate the outgoing light projected into the local area. The receiver captures one or more images of the local area including portions of the outgoing light reflected from the local area. The controller determines depth information for one or more objects in the local area based in part on the captured one or more images.

Non-mechanical beam steering assembly

In one example, an apparatus comprises a plurality of sub-pixels arranged sideway, a shared optical element positioned over the plurality of sub-pixels, the shared optical element being configured to direct light originating from a same location in a scene to each sub-pixel in the plurality of sub-pixels; one or more polarizers positioned between the shared optical element and one or more first sub-pixels of the plurality of sub-pixels and configured to selectively pass one or more components of the light having one or more pre-determined polarization states, to enable the photodiodes of each of the one or more first sub-pixels to generate signals based on intensities of the one or more components; and one or more processors configured to generate polarimetric measurements of the received light based on signals obtained from the photodiodes of the one or more first sub-pixels and polarization properties of the one or more polarizers.

Polarimetric imaging camera

In some examples, an apparatus comprises: a first photodiode to sense a first component of light associated with a first wavelength, and a second photodiode configured to sense a second component of the light associated with a second wavelength, the first component and the second component being associated with, respectively, a first wavelength and a second wavelength. The apparatus further comprises a first optical structure and a second optical structure positioned over, respectively, the first photodiode and the second photodiode. The first optical structure is configured to increase a propagation path of the first component of the light within the first photodiode and has a first optical property based on the first wavelength. The second optical structure is configured to increase a propagation path of the second component of the light within the second photodiode, and has a second optical property based on the second wavelength.

Multi-spectral image sensor

A depth camera assembly (DCA) determines distances between the DCA and objects in a local area within a field of view of the DCA. The DCA projects a series of sinusoidal patterns into the local area DCA and captures images of the sinusoidal patterns via a sensor. Each pixel of the augmented sensor includes a plurality of charge bins, and charge accumulated by a photodiode of a pixel during different time intervals (e.g., times when different sinusoidal patterns are emitted) is stored in a different charge storage bin. Charge may be retrieved from different charge storage bins to determine depth from the DCA.

Timing of multiplexed sensor phase measurements in a depth camera assembly for depth determination using fringe interferometry

A depth camera assembly (DCA) has a light source assembly, a mask, a camera assembly, and a controller. The light source assembly includes at least one light source. The mask is configured to generate an interference pattern that is projected into a target area. The mask has two openings configured to pass through light emitted by the at least one light source, and the light passed through the two openings forms an interference pattern across the target area. The interference pattern has a phase based on a position of the light source. The camera assembly is configured to capture images of a portion of the target area that includes the interference pattern. The controller is configured to determine depth information for the portion of the target area based on the captured images.

Interferometric structured illumination for depth determination

A switchable fringe pattern illuminator includes an optical path switch configured to receive light and dynamically control an amount of light that is provided to a first waveguide and an amount of light that is provided to a second waveguide. A first projector configured to generate a first fringe pattern using light from the first waveguide. The first fringe pattern illuminates a first portion of a target area. A second projector configured to generate a second fringe pattern using light from the second waveguide. The second fringe pattern illuminates a second portion of a target area. The illuminator may be part of a depth camera assembly (DCA). The DCA is configured to capture images of a portion of the target area. The DCA is further configured to determine depth information for an object in the target area based in part on the captured images.

Switchable fringe pattern illuminator

One embodiment of the present disclosure sets forth a method that includes receiving one or more two-dimensional images of one or more light patterns incident on an eye proximate to an eye region of a near-eye display device, and computing a gaze direction associated with the eye based on the one or more two-dimensional images.

Gaze detection pipeline in an artificial reality system

A depth camera assembly (DCA) for depth sensing of a local area. The DCA includes a polarized light generator, an imaging device, and a controller. The polarized light generator modulates one or more optical beams emitted from an illumination source to generate modulated light, and projects the modulated light into the local area as polarized light having a first polarization. The imaging device receives light from the local area, the received light including ambient light and a portion of the polarized light reflected from the local area. The imaging device reduces an intensity of the received light having polarization different from a second polarization to generate filtered light substantially composed of light of the second polarization, and detects the portion of the polarized light having the second polarization using the filtered light. The controller determines depth information for the local area based on the detected portion of the polarized light.

Polarized illumination and detection for depth sensing

Methods and systems for providing illumination for depth sensing are provided. In one example, an apparatus comprises an illuminator, an optical sensor, and a controller. The illuminator comprises a photonic integrated circuit (PIC) and a diffractive optical element (DOE). The PIC including a laser source and at least one waveguide including optical turning features. The at least one waveguide propagates light transmitted by the laser source along a first axis parallel to a top surface of the PIC. The optical turning features diverts the light to form collimated light beams to exit the at least one waveguide and the top surface along a second axis. The DOE can diffract and project the collimated light beams. The optical sensor can detect the collimated light beams reflected off an object. The controller can determine a depth of the object with respect to the apparatus based on the detection.

Photonic integrated circuit illuminator

A depth camera assembly for determining depth information for objects in a local area comprises a light generator, a camera and a controller. The light generator illuminates the local area with structured light in accordance with emission instructions from the controller. The light generator includes an illumination source, an acousto-optic deflector (AOD), and a liquid crystal device (LCD) with liquid crystal gratings (LCGs). The AOD functions as a dynamic diffraction grating that diffracts optical beams emitted from the illumination source to form diffracted scanning beams, based on emission instructions from the controller. Each LCG in the LCD is configured to further diffract light from the AOD to generate the structured light projected into the local area. The camera captures images of portions of the structured light reflected from objects in the local area. The controller determines depth information for the objects based on the captured images.

Ultra-wide field-of-view scanning devices for depth sensing

Non-mechanical beam steering for depth sensing

A compact light projection system is described for use in artificial reality systems, and which outputs patterned interferometric illumination that may be dynamically adjustable. The light projection systems are Integrated Circuits (IC)s, which are compact and easily added to other electronic devices in an artificial reality device. The IC illumination sources described herein provide flexibility by incorporating dynamically adjustable components as well as static components, such as phase delay devices, coupling controllers, switch-able light sources, and output gratings, which may each be adjusted to control the resulting pattern of interferometric illumination.

Growing community of inventors

Redmond, WA, United States of America

Qing Chao