Facebook, Inc.

Facebook Technologies, LLC

Meta Platforms Technologies, LLC

Nitay Romano

Nadav Grossinger

Niv Kantor

Emil Alon

Yair Alpern

Robert Nasry Hasbun

Guy Raz

Ricardo Garcia

Israel Grossinger

Moshe Meyassed

Arnon Gratch

Kobi Kaminitz

Yaakov Yonatan Layosh

In one embodiment, a system includes at least one projector configured to project a plurality of projected patterns, where a projected lighting characteristic of each of the projected patterns varies over a time period in accordance with an associated predetermined temporal lighting-characteristic pattern, a camera configured to capture images of detected patterns during the time period, and one or more processors configured to: determine, for each detected pattern, a detected temporal lighting-characteristic pattern based on variations in a detected lighting characteristic of the detected pattern, identify a detected pattern that corresponds to one of the projected patterns by comparing at least one of the detected temporal lighting-characteristic patterns to at least one of the temporal lighting-characteristic patterns, and compute a depth associated with the detected patterns based on the one or more of the projected patterns, the detected pattern, and a relative position between the camera and the projector.

Depth sensing using temporal coding

In one embodiment, a system includes at least one projector comprising a plurality of light emitters, where the projector is configured to project a projected pattern comprising a plurality of projected features having different locations; a camera configured to capture an image comprising a detected pattern corresponding to a reflection of the projected pattern; and one or more processors configured to: identify at least one detected feature of the detected pattern, wherein the detected feature corresponds to at least one reflection of the projected features; and activate or deactivate one or more of the light emitters based on the detected feature. The light emitters may be activated or deactivated by determining a detected feature measurement based on the detected feature, and activating or deactivating one or more of the light emitters when the detected feature measurement satisfies a threshold feature measurement condition.

Depth sensing using dynamic illumination with range extension

In one embodiment, a system includes at least one projector configured to project a plurality of projected patterns, where a projected lighting characteristic of each of the projected patterns varies over a time period in accordance with an associated predetermined temporal lighting-characteristic pattern, a camera configured to capture images of detected patterns, and one or more processors configured to: determine, for each detected pattern, a detected temporal lighting-characteristic pattern based on variations in a detected lighting characteristic of the detected pattern, identify a detected pattern that corresponds to one of the projected patterns by comparing at least one of the detected temporal lighting-characteristic patterns to at least one of the temporal lighting-characteristic patterns, and compute a depth associated with the detected patterns based on the one or more of the projected patterns, the detected pattern, and a relative position between the camera and the projector.

Apparatus for generating a dynamic structured light pattern for optical tracking in three-dimensional space, comprises an array of lasers, such as a VCSEL laser array, to project light in a pattern into a three-dimensional space; and an optical element or elements arranged in cells. The cells are aligned with subsets of the laser array, and each cell individually applies a modulation, in particular an intensity modulation, to light from the laser or lasers of the subset, to provide a distinguishable and separately controllable part of the dynamic structured light pattern. A method of generating a structured light pattern is disclosed, in which light is provided from an array of lasers, and light is individually projected from subsets of the array of lasers to provide differentiated parts of the structured light pattern.

Three dimensional depth mapping using dynamic structured light

Disclosed herein are techniques for structured light pattern generation. A method for generating a one-dimensional structured light pattern in a first direction and with a desired intensity pattern includes generating a plurality of one-dimensional light patterns using a two-dimensional diffractive optical element with different periods in two orthogonal directions, and combining the plurality of one-dimensional light patterns to form the one-dimensional structured light pattern with the desired intensity pattern. Each of the one-dimensional light patterns includes a one-dimensional light pattern in the first direction. The plurality of one-dimensional light patterns is distributed in a second direction different from the first direction. A separation angle between each pair of adjacent one-dimensional light patterns of the plurality of one-dimensional light patterns in the second direction and with respect to the two-dimensional diffractive optical element is less than a threshold value.

Diffractive optical elements (DOEs) for high tolerance of structured light

In one embodiment, a system includes at least one projector configured to project a plurality of projected patterns onto a scene, the projected patterns including a first projected pattern that includes a plurality of first projected features, a camera configured to capture a plurality of images including a first detected pattern corresponding to a reflection of the first projected pattern, and one or more processors configured to: compute a depth map of the scene based on the first projected pattern, the first detected pattern, and relative positions of the camera and the at least one projector, project, using the projector, a second projected pattern comprising a plurality of second projected features onto a portion of the scene, where the second projected pattern is more sparse than the first projected pattern, and capture, using the camera, a second detected pattern corresponding to a reflection of the second projected pattern.

Low-power depth sensing using dynamic illumination

In one embodiment, a computing system may access a first image and a second image of at least a common portion of an environment while a light emission with a predetermined emission pattern is projected by a projector. The first and second images are respectively captured by a first and a second detector that are respectively separated from the projector by a first and a second distance. The system may determine that a first portion of the first image corresponds to a second portion of the second image. The system may compute, using triangulation, a first depth value associated with the first portion and a second depth value associated with the second portion. The system may determine that the first and second depth values match in accordance with one or more predetermined criteria, and generate a depth map of the environment based on at least one of the depth values.

Detection of structured light for depth sensing

In one embodiment, a system includes a first projector configured to project a first projected pattern having one or more first projected lighting characteristics, a second projector configured to project a second projected pattern having one or more second projected lighting characteristics, a camera configured to capture an image comprising first and second detected patterns corresponding to reflections of the first and second projected patterns, respectively, and one or more processors configured to: identify a detected point in the image that corresponds to a projected point in at least one of the first and second projected patterns by comparing detected lighting characteristics of the first and second detected patterns with the first and second projected lighting characteristics, and compute a depth associated with the detected point based on the projected point, the detected point, and a relative position between the camera and at least one of the projectors.

Depth sensing using grid light patterns

Apparatus for structured light scanning. The structured light includes one or more projected lines or other patterns. At least two independent emitters emit light for each projected line or pattern. Typically the at least two independent emitters are arranged in a row. The apparatus also includes a pattern generator for causing light from respective emitters of a given row to overlap along a pattern axis to form a projected pattern.

Structured light sensing for 3D sensing

A device connectable to a near eye display or to a virtual reality headset and to a portable computing platform having a processor is provided herein. The device may include: an illuminator configured to illuminate a proximity of a user wearing the headset or the near eye display with patterned light; and an TR camera configured to capture reflections of said patterned light coming from at least one object located in the proximity of the user, wherein the processor is configured to: establish data and power connection between the device and said poliable computing platform and said near eye display; and generate a depth map of said object based on the reflections.

System, device and method for providing user interface for a virtual reality environment

An object is identified or tracked within a volume by projecting a light beam encoded with one or more predefined properties to have a predefined optical structure into the volume. A detector captures light from the predefined optical structure reflected from the volume. By analyzing one or more characteristics of light from the predefined optical structure reflected from the object, the object is segmented from the volume.

Growing community of inventors

Geva Binyamin, Israel

Nitay Romano