Facebook Technologies, LLC

Meta Platforms Technologies, LLC

Fenwick & West Llp

Fengqiang Li

Michael Anthony Hall

Zihe Gao

Xinqiao Liu

Nicholas Daniel Trail

Zhaoming Zhu

Shuochen Su

Huixuan Tang

A depth camera assembly (DCA) optimizes illumination and image capture of a local area to generate depth information of the local area. The DCA determines depth information for a first portion of the local area viewable at a first pose. The DCA is moved from the first pose to a second pose, where a second portion of the local area is viewable and overlaps with the first portion. The overlapping region is not illuminated by the DCA. A non-overlapping portion of the second portion is illuminated, captured, and depth information determined.

Dynamic illumination control for depth determination

A camera assembly for determining depth information for a local area includes a light source assembly, a camera assembly, and a controller. The light source assembly projects pulses of light into the local area. The camera assembly images a portion of the local area illuminated with the pulses. The camera assembly includes augmented pixels, each augmented pixel having a plurality of gates and at least some of the gates have a respective local storage location. An exposure interval of each augmented pixel is divided into intervals associated with the gates, and each local storage location stores image data during a respective interval. The controller reads out, after the exposure interval of each augmented pixel, the image data stored in the respective local storage locations of each augmented pixel to generate image data frames. The controller determines depth information for the local area based in part on the image data frames.

Depth determination using time-of-flight and camera assembly with augmented pixels

A camera assembly for generating high dynamic range images. The camera assembly includes a sensor that images a portion of a local area, and a controller. The sensor includes a plurality of augmented pixels, each augmented pixel having a plurality of gates and at least some of the gates have a respective local storage location. An exposure interval of each augmented pixel is divided into intervals associated with the gates, and each local storage location stores image data during a respective interval. The controller reads out, after the exposure interval of each augmented pixel, the image data stored in the respective local storage locations of each augmented pixel to form intermediate images that each have a dynamic range. The controller then generates an image for the portion of the local area using the intermediate images, the image having a higher dynamic range than each of the intermediate images.

High dynamic range camera assembly with augmented pixels

A camera assembly for depth sensing of a local area. The camera assembly includes a projector, a sensor and a controller. The projector emits, into the local area, a structured light (SL) pattern at multiple phases. The sensor images the local area using multiple augmented pixels. Each augmented pixel has multiple gates and at least some of the gates have a respective local storage location. Each capture phase of the augmented pixel is divided into a respective plurality of time bins associated with a respective subset of the gates. Each local storage location of the subset of gates stores image data during a respective time bin, the image data based on portions of the SL pattern emitted at a respective phase and reflected from the local area. The controller determines depth information for the local area based on the image data stored during at least one capture phase.

Three-dimensional imaging with spatial and temporal coding for depth camera assembly

A camera captures image data at a target frame rate. The camera includes a sensor and a controller. The sensor is configured to detect light from a local area and includes a plurality of augmented pixels. Each augmented pixel comprises at least a first and a second gate. The first gates are configured to store a first plurality of image frames as first image data according to a first activation pattern. The second gates are configured to store a second plurality of image frames as second image data according to a second activation pattern. The controller reads out the image data to generate a first image from the first image data and a second image from the second image data. The first and second images may be used to reconstruct a combined set of image frames at the target frame rate with a reconstruction algorithm.

High frame rate reconstruction with N-tap camera sensor

A calibration assembly for calibrating a depth sensor. The calibration assembly includes a pathway and a controller coupled to a depth sensor. The pathway is coupled to a light source, and a pulse of light from the light source is directed through the pathway to the depth sensor at a first time. The controller determines a second time at which a measurement of the light pulse detected at an augmented pixel of the depth sensor is read out. The controller computes a readout delay for the augmented pixel based on the first time, the second time, and an estimated travel time for the light pulse from the light source to the augmented pixel. The controller stores the readout delay, and a depth camera that includes the depth sensor is configured to calibrate out the readout delay relative to other augmented pixels of the depth sensor.

Calibration apparatus for time of flight depth sensor

A camera assembly for generating differential images for a local area. The camera assembly includes a sensor and a controller. The sensor images at least a portion of a local area and includes a plurality of augmented pixels. Each augmented pixel has at least a first gate with a first local storage location and a second gate with a second local storage location. An exposure interval of the sensor is divided into intervals. The first local storage location stores first image data during a first of the intervals, and the second local storage location stores second image data during a second of the intervals. The controller determines, for each augmented pixel, a difference between the first image data and the second image data. The controller then reads out the difference for each augmented pixel as a differential image of the portion of the local area.

Differentiated imaging using camera assembly with augmented pixels

A depth camera assembly (DCA) for depth sensing of a local area. The DCA includes a light source assembly, a camera assembly, and a controller. The light source assembly projects pulses of light into the local area, each pulse having a respective polarization type. The camera assembly images a portion of the local area illuminated with the pulses of light. The camera assembly includes a plurality of augmented pixels, each augmented pixel having a plurality of gates and local storage locations. An exposure interval of the camera assembly is divided into intervals and some of the intervals are synchronized to the projected pulses of light such that each respective local storage location stores image data associated with a different polarization type. The controller determines depth information for the local area based in part on the polarization types associated with the image data stored in respective local storage locations.

Growing community of inventors

Redmond, WA, United States of America

Fengqiang Li