Facebook Technologies, LLC

Meta Platforms Technologies, LLC

Joseph Duggan

Fei Liu

Mark Timothy Sullivan

Andrew Matthew Bardagjy

Simon Morris Shand Weiss

Cina Hazegh

A depth camera assembly (DCA) for depth sensing of a local area includes a structured light generator, an imaging device, and a controller. The structured light generator illuminates the local area with a structured light pattern. The structured light generator includes a programmable diffractive optical element (PDOE) that generates diffracted scanning beams using optical beams. The PDOE functions as a dynamic diffraction grating that dynamically adjusts diffraction of the optical beams to generate the diffracted scanning beams of different patterns. The diffracted scanning beams are projected as the structured light pattern into the local area, wherein the structured light pattern is dynamically adjustable based on the PDOE. The imaging device captures image(s) of at least a portion of the structured light pattern reflected from object(s) in the local area. The controller determines depth information for the object(s) based on the captured image(s).

Camera assembly with programmable diffractive optical element for depth sensing

A depth camera assembly for depth sensing of a local area includes a structured light generator, an imaging device, and a controller. The structured light generator illuminates a local area with structured light and includes an illumination source, a mounting element, and a diffractive optical element (DOE) mounted on the mounting element. The mounting element can have a plurality of adjustable positions relative to the illumination source based on emission instructions from the controller. The DOE generates, using light emitted from the illumination source, diffracted scanning beams that are projected as the structured light into the local area. A pattern of the structured light is dynamically adjustable and unique for each adjustable position of the mounting element. The imaging device captures image(s) of the structured light reflected from object(s) in the local area. The controller determines depth information for the object(s) based in part on the captured image(s).

Shifting diffractive optical element for adjustable depth sensing resolution

An imaging device for imaging of a local area surrounding the imaging device. The imaging device includes a lens assembly, a filtering element and a detector. The lens assembly is configured to receive light from a local area surrounding the imaging device and to direct at least a portion of the received light to the detector. The filtering element is placed in the imaging device within the lens assembly such that light is incident at a surface of the filtering element within a range of angles determined by a design range of angles at which the filtering element is designed to filter light. The detector is configured to capture image(s) of the local area including the filtered light. The imaging device can be integrated into a depth camera assembly for determining depth information of object(s) in the local area based on the captured image(s).

Imaging device based on lens assembly with embedded filter

A head-mounted display (HMD) system includes a projector system configured to emit a structured light (SL) pattern onto one or more objects in a local area, the projected SL pattern comprises at least a first SL pattern having a first field of view (FOV) corresponding to a first tileable boundary, and a second SL pattern having a second FOV corresponding to a second tileable boundary. Each of the projected SL patterns contains fiducials at predetermined locations within the SL pattern. An imaging device captures images of the local area including at least portions of the first SL pattern or the second SL patterns. The detected locations of the fiducials in the captured images is used to determine the boundary locations of the first and second SL patterns. Depth information of the local area is generated based upon the determined locations of the first and second SL patterns.

Tileable structured light projection system

A depth measurement assembly (DMA) measures depth information of an object in a local area. The DMA includes structured light projector, a depth camera assembly, and a controller. The structured light projector projects structured light patterns into the local area. The structured light projector includes a diffractive optical unit that includes diffractive optical elements (DOEs) and selects a DOE. The selected DOE is illuminated by light from a light source and converts the light into a structured light pattern. In some embodiment, the diffractive optical units selects multiple DOEs associated with multiple structured light patterns. The structured light pattern is projected into the local area by a projection assembly of the structured light projector and illuminates the object. The depth camera assembly captures images of the object. The controller uses the captured images to determine depth information of the object.

Depth measurement using scanning diffractive optical elements

A camera system configured to generate depth information for a local area. The camera system comprises a plurality of depth camera sub-assemblies arranged in a substantially spherical arrangement. Each sub-assembly comprises a projector that projects a structured light pattern onto a portion of the local area, such that the projected light patterns of the plurality of sub-assemblies form a tiled light pattern covering 360 degrees of the local area. Each sub-assembly further comprises at least one camera is configured to capture images of the local area. A controller of the camera system is configured to receive the captured images and to construct a 360 degree depth map of the scene, based upon the structured light patterns projected by the projectors of the plurality captured in the received images.

3-D 360 degrees depth projector

A wavelength tuning system determines a temperature calibrated to a DOE of a structured light (SL) projector. The wavelength tuning system includes a camera and controller. The camera captures images of a SL pattern projected by the SL projector. The controller generates tuning instruction. The tuning instructions cause a wavelength regulator of the SL projector to set a light source of the SL projector to different temperatures. The tuning instruction also cause the camera to capture images of the structured light pattern at each of the different temperatures. Using at least some of the captured images, the controller determines the temperature calibrated to the DOE. In one embodiment, the temperature calibrated to the DOE corresponds to a wavelength of light emitted by the light source that result in an estimated minimum power of a zeroth order diffracted beam of the SL pattern.

Wavelength tuning for diffractive optical elements of structured light projectors

A head mounted display (HMD) displays content to a user wearing the HMD, where the content may be based on a facial model of the user. The HMD uses an electronic display to illuminate a portion of the face of the user with. The electronic display emits a pattern of structured light and/or monochromatic light of a given color. A camera assembly captures images of the illuminated portion of the face. A controller processes the captured images to determine depth information or color information of the face of the user. Further, the processed images may be used to update the facial model, for example, which is represented as a virtual avatar and presented to the user in a virtual reality, augmented reality, or mixed reality environment.

Face tracking using structured light within a head-mounted display

An optical characterization system includes a camera assembly and a workstation. The camera assembly is configured to capture images of different portions of a structured light pattern emitted from a device under test in accordance with imaging instructions. In some embodiments, the device under test may be a diffractive optical element (DOE). The workstation provides the imaging instructions to the camera assembly, and stitch the captured images together to form a pattern image. The pattern image is a single image of the entire structured light pattern. The workstation also characterizes performance of the device under test using the pattern image and a performance metric.

Growing community of inventors

San Francisco, CA, United States of America

Joseph Duggan