Qualcomm Incorporated

Albrecht Johannes Lindner

Kalin Mitkov Atanassov

James Wilson Nash

Stephen Michael Verrall

Hasib Siddiqui

Volodimir Slobodyanyuk

Sergiu Radu Goma

Htet Naing

Evgeni Petrovich Gousev

Russell Wayne Gruhlke

Khurshid Syed Alam

Vikas Ramachandra

John Michael Wyrwas

Robert Mitchell Kleiman

Hasib Ahmed Siddiqui

An electronic imaging device and method for image capture are described. The imaging device includes a camera configured to obtain image information of a scene and that may be focused on a region of interest in the scene. The imaging device also includes a LIDAR unit configured to obtain depth information of at least a portion of the scene at specified scan locations of the scene. The imaging device is configured to detect an object in the scene and provides specified scan locations to the LIDAR unit. The camera is configured to capture an image with an adjusted focus based on depth information, obtained by the LIDAR unit, associated with the detected object.

Systems and methods for depth map sampling

Aspects of the present disclosure relate to systems and methods for structured light depth systems. An example active depth system may include a receiver to receive reflections of transmitted light and a transmitter including one or more light sources to transmit light in a spatial distribution. The spatial distribution of transmitted light may include a first region of a first plurality of light points and a second region of a second plurality of light points. A first density of the first plurality of light points is greater than a second density of the second plurality of light points when a first distance between a center of the spatial distribution and a center of the first region is less than a second distance between the center of the spatial distribution and the center of the second region.

Light distribution for active depth systems

Aspects of the present disclosure relate to systems and methods for determining one or more depths. An example system includes a time-of-flight receiver configured to sense pulses from reflections of light from a structured light transmitter. An example method includes sensing, by a time-of-flight receiver, pulses from reflections of light from a structured light transmitter.

Time of flight range finder for a structured light system

A method and electronic device for selectively obtaining depth information at locations within a scene are described. Image content analysis is performed on a received image of the scene. Locations within the image to obtain depth information within the scene are determined based on the image analysis. The locations are provided to a LIDAR (light+radar) unit to selectively obtain depth information at the scene locations. The depth information is received from the LIDAR unit and a second image analysis is performed on a second image. The second image analysis is based on the image content of the second image and the received depth information. Updated locations based on the second image analysis may be provided to the LIDAR unit to obtain updated depth information.

Systems, methods, and devices for generating a depth map of a scene are provided. The method comprises projecting, onto the scene, a codeword pattern including a plurality of light points including a first set of light points projected at a first intensity and a second set of light points projected at a second intensity greater than the first intensity. The method further comprises detecting, from the scene, a reflection of the codeword pattern. The method further comprises generating a first depth map layer at a first resolution. The method further comprises generating a second depth map layer at a second resolution lower than the first resolution. The method further comprises generating the depth map of the scene, wherein the depth map includes depth information for the scene according to each of the first depth map layer and the second depth map layer.

Systems and methods for generating a structured light depth map with a non-uniform codeword pattern

Systems and methods for reconstructing an object boundary in a disparity map generated by a structured light system are disclosed. One aspect is a structured light system. The system includes an image projecting device configured to project codewords. The system further includes a receiver device including a sensor, the receiver device configured to sense the projected codewords reflected from an object. The system further includes a processing circuit configured to generate a disparity map of the object, detect a first boundary of the object in the disparity map, identify a shadow region in the disparity map adjoining the first boundary, the shadow region including pixels with codeword outages, and change a shape of the object in the disparity map based on the detected shadow region. The system further includes a memory device configured to store the disparity map.

Object reconstruction in disparity maps using displaced shadow outlines

A device includes a first camera and a processor configured to detect one or more first keypoints within a first image captured by the first camera at a first time, detect one or more second keypoints within a second image captured by a second camera at the first time, and detect the one or more first keypoints within a third image captured by the first camera at a second time. The processor is configured to determine a pose estimation based on coordinates of the one or more first keypoints of the first image relative to a common coordinate system, coordinates of the one or more second keypoints of the second image relative to the common coordinate system, and coordinates of the one or more first keypoints of the third image relative to the common coordinate system. The first coordinate system is different than the common coordinate system.

Pose estimation using multiple cameras

Methods, systems, and apparatuses are provided to compensate for a misalignment of optical devices within an imaging system. For example, the methods receive image data captured by a first optical device having a first optical axis and a second optical device having a second optical axis. The methods also receive sensor data indicative of a deflection of a substrate that supports the first and second optical devices. The deflection can result from a misalignment of the first optical axis relative to the second optical axis. The methods generate a depth value based on the captured image data and the sensor data. The depth value can reflect a compensation for the misalignment of the first optical axis relative to the second optical axis.

Active alignment correction for optical systems

Systems, apparatus, and methods for generating a fused depth map from one or more individual depth maps, wherein the fused depth map is configured to provide robust depth estimation for points within the depth map. The methods, apparatus, or systems may comprise components that identify a field of view (FOV) of an imaging device configured to capture an image of the FOV and select a first depth sensing method. The system or method may sense a depth of the FOV with respect to the imaging device using the first selected depth sensing method and generate a first depth map of the FOV based on the sensed depth of the first selected depth sensing method. The system or method may also identify a region of one or more points of the first depth map having one or more inaccurate depth measurements and determine if additional depth sensing is needed.

Method and apparatus for multiple technology depth map acquisition and fusion

An electronic device for generating a corrected depth map is described. The electronic device includes a processor. The processor is configured to obtain a first depth map. The first depth map includes first depth information of a first portion of a scene sampled by the depth sensor at a first sampling. The processor is also configured to obtain a second depth map. The second depth map includes second depth information of a second portion of the scene sampled by the depth sensor at a second sampling. The processor is additionally configured to obtain displacement information indicative of a displacement of the depth sensor between the first sampling and the second sampling. The processor is also configured to generate a corrected depth map by correcting erroneous depth information based on the first depth information, the second depth information, and the displacement information.

Systems and methods for correcting erroneous depth information

An electronic device is described. The electronic device includes a camera configured to capture an image of a scene. The electronic device also includes an image segmentation mapper configured to perform segmentation of the image based on image content to generate a plurality of image segments, each of the plurality of image segments associated with spatial coordinates indicative of a location of each segment in the scene. The electronic device further includes a memory configured to store the image and the spatial coordinates. The electronic device additionally includes a LIDAR (light+radar) unit, the LIDAR unit steerable to selectively obtain depth values corresponding to at least a subset of the spatial coordinates. The electronic device further includes a depth mapper configured to generate a depth map of the scene based on the depth values and the spatial coordinates.

Systems and methods configured to generate virtual gimbal information for range images produced from 3D depth scans are described. In operation according to embodiments, known and advantageous spatial geometries of features of a scanned volume are exploited to generate virtual gimbal information for a pose. The virtual gimbal information of embodiments may be used to align a range image of the pose with one or more other range images for the scanned volume, such as for combining the range images for use in indoor mapping, gesture recognition, object scanning, etc. Implementations of range image registration using virtual gimbal information provide a realtime one shot direct pose estimator by detecting and estimating the normal vectors for surfaces of features between successive scans which effectively imparts a coordinate system for each scan with an orthogonal set of gimbal axes and defines the relative camera attitude.

Registration of range images using virtual gimbal information

Systems and methods for controlling structured light laser systems are disclosed. One aspect is a structured light system. The system includes a memory device configured to store a depth map. The system further includes an image projecting device including a laser system configured to project codewords. The system further includes a receiver device including a sensor, the receiver device configured to sense the projected codewords reflected from an object. The system further includes a processing circuit configured to retrieve a portion of the depth map and calculate expected codewords from the depth map. The system further includes a feedback system configured to control the output power of the laser system based on the sensed codewords and the expected codewords.

Code domain power control for structured light

Methods and systems for surface normal estimation are disclosed. In some aspects, a plurality of images or depth maps representing a three dimensional object from multiple viewpoints is received. Surface normals at surface points within a single image of the plurality of images are estimated based on surface points within the single image. An electronic representation of a three dimensional surface of the object is generated based on the surface normals and a point cloud comprised of surface points derived from the plurality of images.

Systems and methods for improved surface normal estimation

An electronic device for selecting a transform is described. The electronic device includes at least one image sensor, a memory, and a processor coupled to the memory and to the at least one image sensor. The processor is configured to obtain at least two images from the at least one image sensor. The processor is also configured to characterize structural content of each of the at least two images to produce a characterization for each image that is relevant to transform performance. The processor is further configured to select at least one transform from a set of transforms based on the characterization. The processor is additionally configured to apply the at least one transform to at least one of the images to substantially align the at least two images.

Growing community of inventors

La Jolla, CA, United States of America

Albrecht Johannes Lindner