Facebook Technologies, LLC

Meta Platforms Technologies, LLC

Zihe Gao

Michael Anthony Hall

Zhaoming Zhu

Xinqiao Liu

Fengqiang Li

Shuochen Su

Huixuan Tang

Rajesh Lachhmandas Chhabria

Nicholas Daniel Trail

Qing Chao

Tin Komljenovic

Nadav Grossinger

Robert Nasry Hasbun

Guohua Wei

Jonatan Ginzburg

Michael Hall

A depth camera assembly (DCA) optimizes illumination and image capture of a local area to generate depth information of the local area. The DCA determines depth information for a first portion of the local area viewable at a first pose. The DCA is moved from the first pose to a second pose, where a second portion of the local area is viewable and overlaps with the first portion. The overlapping region is not illuminated by the DCA. A non-overlapping portion of the second portion is illuminated, captured, and depth information determined.

Dynamic illumination control for depth determination

A depth camera assembly (DCA) determines depth information for a local area. The DCA includes a camera assembly and at least one illuminator. The DCA may select a subset of the VCSELs to provide illumination at any given time. The illuminator may comprise near-field VCSELs configured to generate a structured light (SL) pattern for depth sensing in the near-field and far-field VCSELs configured to generate a SL pattern for depth sensing in the far-field. The near-field VCSELs may comprise a linear emission region which is shorter than a linear emission region of the far-field VCSELs. The DCA may generate and phase shift a quasi-sinusoidal SL pattern. The DCA may phase shift the quasi-sinusoidal SL pattern by alternating which traces on the illuminator are active.

VCSEL arrays for generation of linear structured light features

A depth camera assembly (DCA) determines depth information. The DCA projects a dynamic structured light pattern into a local area and captures images including a portion of the dynamic structured light pattern. The DCA determines regions of interest in which it may be beneficial to increase or decrease an amount of texture added to the region of interest using the dynamic structured light pattern. For example, the DCA may identify the regions of interest based on contrast values calculated using a contrast algorithm, or based on the parameters received from a mapping server including a virtual model of the local area. The DCA may selectively increase or decrease an amount of texture added by the dynamic structured light pattern in portions of the local area. By selectively controlling portions of the dynamic structured light pattern, the DCA may decrease power consumption and/or increase the accuracy of depth sensing measurements.

Dynamic adjustment of structured light for depth sensing systems based on contrast in a local area

A switchable fringe pattern illuminator includes an optical path switch configured to receive light and dynamically control an amount of light that is provided to a first waveguide and an amount of light that is provided to a second waveguide. A first projector configured to generate a first fringe pattern using light from the first waveguide. The first fringe pattern illuminates a first portion of a target area. A second projector configured to generate a second fringe pattern using light from the second waveguide. The second fringe pattern illuminates a second portion of a target area. The illuminator may be part of a depth camera assembly (DCA). The DCA is configured to capture images of a portion of the target area. The DCA is further configured to determine depth information for an object in the target area based in part on the captured images.

Switchable fringe pattern illuminator

A photonic integrated circuit assembly generates light for use in a display system or a depth sensing system. The photonic integrated circuit assembly includes optical conditioning elements integrated into a photonic integrated circuit. The photonic integrated circuit includes one or more light sources on the photonic integrated circuit. The photonic integrated circuit includes a waveguide and one or more gratings located in a core of the photonic integrated circuit. The gratings may collimate and/or collocate light emitted by the light source. The photonic integrated circuit may include a beam shaping element integrated into the photonic integrated circuit. A MEMS scanner may use the collocated and/or collimated light emitted by the photonic integrated circuit to generate a display for a user or to generate a structured light pattern for use in a depth sensing system.

Photonic integrated circuits with integrated optical conditioning elements

A camera assembly for depth sensing of a local area. The camera assembly includes a projector, a sensor and a controller. The projector emits, into the local area, a structured light (SL) pattern at multiple phases. The sensor images the local area using multiple augmented pixels. Each augmented pixel has multiple gates and at least some of the gates have a respective local storage location. Each capture phase of the augmented pixel is divided into a respective plurality of time bins associated with a respective subset of the gates. Each local storage location of the subset of gates stores image data during a respective time bin, the image data based on portions of the SL pattern emitted at a respective phase and reflected from the local area. The controller determines depth information for the local area based on the image data stored during at least one capture phase.

Three-dimensional imaging with spatial and temporal coding for depth camera assembly

A camera captures image data at a target frame rate. The camera includes a sensor and a controller. The sensor is configured to detect light from a local area and includes a plurality of augmented pixels. Each augmented pixel comprises at least a first and a second gate. The first gates are configured to store a first plurality of image frames as first image data according to a first activation pattern. The second gates are configured to store a second plurality of image frames as second image data according to a second activation pattern. The controller reads out the image data to generate a first image from the first image data and a second image from the second image data. The first and second images may be used to reconstruct a combined set of image frames at the target frame rate with a reconstruction algorithm.

High frame rate reconstruction with N-tap camera sensor

Dynamic structured light for depth sensing systems based on contrast in a local area

A depth camera assembly (DCA) determines depth information within a local area by capturing images of the local area including a local region using a plurality of imaging sensors. The local region is represented by a first set of pixels in each captured image. For each image, the DCA identifies the first set of pixels corresponding to the surface in the local region and determines a depth measurement from the DCA to the local region by comparing the first set of pixels from images captured by different imaging sensors. To determine depth measurements for second sets of pixels neighboring the first set of pixels, the DCA selectively propagates depth information from the first set of pixels to second sets of pixels satisfying one or more criteria (e.g., satisfying a threshold saturation measurement or a threshold contrast measurement).

Growing community of inventors

Redmond, WA, United States of America

Zihe Gao