Facebook Technologies, LLC

Meta Platforms Technologies, LLC

Rajesh Lachhmandas Chhabria

Michael Anthony Hall

Shuochen Su

Xinqiao Liu

Zhaoming Zhu

Huixuan Tang

Zihe Gao

Steven John Lovegrove

David Vakrat

Monica Stewart

Shengqiong Xie

Yongjun Kim

Zijian Wang

James Steven Hegarty

Michael Hall

A depth camera assembly (DCA) determines depth information. The DCA projects a dynamic structured light pattern into a local area and captures images including a portion of the dynamic structured light pattern. The DCA determines regions of interest in which it may be beneficial to increase or decrease an amount of texture added to the region of interest using the dynamic structured light pattern. For example, the DCA may identify the regions of interest based on contrast values calculated using a contrast algorithm, or based on the parameters received from a mapping server including a virtual model of the local area. The DCA may selectively increase or decrease an amount of texture added by the dynamic structured light pattern in portions of the local area. By selectively controlling portions of the dynamic structured light pattern, the DCA may decrease power consumption and/or increase the accuracy of depth sensing measurements.

Dynamic adjustment of structured light for depth sensing systems based on contrast in a local area

A depth estimation system is described capable of determining depth information using two images from two cameras. A first camera captures a first image and a second camera captures a second image, both images including a plurality of light channels. A scan direction is selected from a plurality of scan directions. For the selected scan direction, along each of a plurality of scanlines, the system compares pixels from the first image to pixels from the second image. The comparison is based on calculating a census transform for each pixel in the first image and a census transform for each pixel in the second image. This comparison is used to determine a stereo correspondence between the pixels in the first image and the pixels in the second image. The system generates a depth map based on the stereo correspondence.

Multi-channel depth estimation using census transforms

A method for depth sensing from an image of a projected pattern is performed at an electronic device with one or more processors and memory. The method includes receiving an image of a projection of an illumination pattern; for a portion of the image, selecting a candidate image of a plurality of candidate images by comparing the portion of the image with a plurality of candidate images; and determining a depth for the portion of the image based on depth information associated with the selected candidate image. Related electronic devices and computer readable storage medium are also disclosed.

Methods for depth sensing using candidate images selected based on an epipolar line

An electronic device includes one or more processors and memory storing instructions for execution by the one or more processors. The stored instructions include instructions for: receiving infrared image information for a three-dimensional area; receiving non-infrared image information for the same three-dimensional area; performing nonlinear intensity adjustment for the received infrared image information; performing nonlinear intensity adjustment for the received non-infrared image information; blending the intensity-adjusted infrared image information and the intensity-adjusted non-infrared image information to obtain a merged image information; and providing the merged image information for determining a depth map. Also disclosed are a corresponding method performed by the electronic device and a computer readable storage medium storing instructions for execution by one or more processors of an electronic device.

Infrared and non-infrared channel blender for depth mapping using structured light

Dynamic structured light for depth sensing systems based on contrast in a local area

A depth estimation system is described capable of determining depth information using two images from two cameras. A first camera captures a first image and a second camera captures a second image, both images including a plurality of light channels. In a first light channel of the plurality of light channels, the system calculates a census transform for each pixel of the first image and a census transform for each pixel of the second image. In a second light channel of the plurality of light channels, the system calculates a census transform for each pixel of the first image and a census transform for each pixel of the second image. The system generates a depth map based in part on the census transforms for each pixel of the first image and the second image in the first light channel and in the second light channel.

A depth camera assembly (DCA) determines depth information within a local area. The DCA may selectively process a subset of data captured by an imaging sensor and obtained from the imaging sensor, such as pixels corresponding to a region of interest, for depth information. Alternatively, the DCA may limit retrieval of data from the imaging sensor to pixels corresponding to the region of interest from the imaging sensor for processing for depth information. The depth processing may include a semi-global match (SGM) algorithm, and the DCA adjusts a number of neighboring pixels used for determining depth information for a specific pixel based on one or more criteria. In some embodiments, the DCA performs the depth processing by analyzing images from different image sensors using left to right and right to left correspondence checks that are performed in parallel.

Selective processing or readout of data from one or more imaging sensors included in a depth camera assembly

A depth camera assembly (DCA) determines depth information within a local area by capturing images of the local area including a local region using a plurality of imaging sensors. The local region is represented by a first set of pixels in each captured image. For each image, the DCA identifies the first set of pixels corresponding to the surface in the local region and determines a depth measurement from the DCA to the local region by comparing the first set of pixels from images captured by different imaging sensors. To determine depth measurements for second sets of pixels neighboring the first set of pixels, the DCA selectively propagates depth information from the first set of pixels to second sets of pixels satisfying one or more criteria (e.g., satisfying a threshold saturation measurement or a threshold contrast measurement).

Growing community of inventors

San Jose, CA, United States of America

Rajesh Lachhmandas Chhabria