Qualcomm Incorporated

Stephen Michael Verrall

Kalin Mitkov Atanassov

Albrecht Johannes Lindner

James Wilson Nash

Volodimir Slobodyanyuk

Shandon Campbell

Sergiu Radu Goma

Hasib Siddiqui

Ovidiu Cristian Miclea

Kuo-Chun Lee

Shailesh Maheshwari

Sivaramakrishna Veerepalli

Nischal Abrol

Thadi Manjunath Nagaraj

Ron Keidar

Jack Shauh

Sandeep Singhai

Karthikeyan Shanmugavadivelu

Htet Naing

Nela Gurevich

Yunke Pan

Rajesh Narayanan

Giridhar Kapalli

Constantine Elster

Sangyeol Kang

Adrian Giura

Hasib Ahmed Siddiqui

An electronic imaging device and method for image capture are described. The imaging device includes a camera configured to obtain image information of a scene and that may be focused on a region of interest in the scene. The imaging device also includes a LIDAR unit configured to obtain depth information of at least a portion of the scene at specified scan locations of the scene. The imaging device is configured to detect an object in the scene and provides specified scan locations to the LIDAR unit. The camera is configured to capture an image with an adjusted focus based on depth information, obtained by the LIDAR unit, associated with the detected object.

Systems and methods for depth map sampling

Aspects of the present disclosure relate to systems and methods for structured light depth systems. An example active depth system may include a receiver to receive reflections of transmitted light and a transmitter including one or more light sources to transmit light in a spatial distribution. The spatial distribution of transmitted light may include a first region of a first plurality of light points and a second region of a second plurality of light points. A first density of the first plurality of light points is greater than a second density of the second plurality of light points when a first distance between a center of the spatial distribution and a center of the first region is less than a second distance between the center of the spatial distribution and the center of the second region.

Light distribution for active depth systems

A method and electronic device for selectively obtaining depth information at locations within a scene are described. Image content analysis is performed on a received image of the scene. Locations within the image to obtain depth information within the scene are determined based on the image analysis. The locations are provided to a LIDAR (light+radar) unit to selectively obtain depth information at the scene locations. The depth information is received from the LIDAR unit and a second image analysis is performed on a second image. The second image analysis is based on the image content of the second image and the received depth information. Updated locations based on the second image analysis may be provided to the LIDAR unit to obtain updated depth information.

Aspects of the present disclosure relate to systems and methods for time-of-flight ranging. An example time-of-flight system includes a transmitter including a plurality of light emitters for transmitting focused light, the plurality of light emitters including a first group of light emitters for transmitting focused light with a first field of transmission and a second group of light emitters for transmitting focused light with a second field of transmission. The first field of transmission at a depth from the transmitter is larger than the second field of transmission at the depth from the transmitter. The time-of-flight system also includes a receiver to receive reflections of the transmitted light.

Time of flight ranging with varying fields of emission

Aspects of the present disclosure relate to active depth sensing. An example device may include a memory and a processor coupled to the memory. The processor may be configured to determine an amount of ambient light for a scene to be captured by a structured light receiver and adjust an exposure time for frame capture of a sensor of the structured light receiver based on the determined amount of ambient light. The exposure time of the sensor of the structured light receiver is inversely related to the determined amount of ambient light. The processor further may be configured to receive a plurality of captured frames from the structured light receiver based on the adjusted exposure time and generate an aggregated frame, including aggregating values across the plurality of captured frames.

Adjustable receiver exposure times for active depth sensing systems

Methods and systems disclosed provide improved depth sensing for an imaging device. In some aspects, a plurality of transmitters including a first transmitter are configured to generate a structured light pattern with a first depth of field at a first resolution and a second structured light pattern with a second depth of field at the first resolution, wherein the second depth of field is wider than the first depth of field. A receiver, such as an imaging sensor, is configured to focus within the first depth of field to receive the first structured light pattern and capture a first image representing the first structured light pattern and to focus within the second depth of field to receive the second structured light pattern and capture a second image representing the second structured light pattern. An electronic hardware processor is configured to generate a depth map of the scene based on the first image and the second image.

Systems and methods for improved depth sensing

An electronic device for generating a corrected depth map is described. The electronic device includes a processor. The processor is configured to obtain a first depth map. The first depth map includes first depth information of a first portion of a scene sampled by the depth sensor at a first sampling. The processor is also configured to obtain a second depth map. The second depth map includes second depth information of a second portion of the scene sampled by the depth sensor at a second sampling. The processor is additionally configured to obtain displacement information indicative of a displacement of the depth sensor between the first sampling and the second sampling. The processor is also configured to generate a corrected depth map by correcting erroneous depth information based on the first depth information, the second depth information, and the displacement information.

Systems and methods for correcting erroneous depth information

An electronic device is described. The electronic device includes a camera configured to capture an image of a scene. The electronic device also includes an image segmentation mapper configured to perform segmentation of the image based on image content to generate a plurality of image segments, each of the plurality of image segments associated with spatial coordinates indicative of a location of each segment in the scene. The electronic device further includes a memory configured to store the image and the spatial coordinates. The electronic device additionally includes a LIDAR (light+radar) unit, the LIDAR unit steerable to selectively obtain depth values corresponding to at least a subset of the spatial coordinates. The electronic device further includes a depth mapper configured to generate a depth map of the scene based on the depth values and the spatial coordinates.

Systems and methods of triggering an event based on meeting a certain depth criteria in an image. One innovation of a method includes a identifying at least one object in a field of view of an imaging device, the imaging device configured to capture at least one image of the field of view, determining a threshold depth level, measuring a depth of the at least one object within the field of view with respect to the imaging device, comparing the measured depth of the at least one object to the threshold depth level, and capturing an image of the object when the depth of the object within the field of view exceeds the threshold depth level.

Depth triggered event feature

A structured light active sensing systems may be configured to transmit and received codewords to generate a depth map by analyzing disparities between the locations of the transmitted and received codewords. To determine the locations of received codewords, an image of the projected codewords is identified, from which one or more codeword boundaries are detected. The codeword boundaries may be detected based upon a particular codeword bit of each codeword. Each detected codeword boundary may be constrained from overlapping with other detected codeword boundaries, such that no pixel of the received image is associated with more than one codeword boundary.

Methods and apparatus for codeword boundary detection for generating depth maps

Aspects disclosed herein relate to aggregating functionality of computer machine instructions to generate additional computer machine instructions and including the additional computer machine instructions in an instruction set architecture (ISA). An exemplary method includes selecting at least first and second computer machine instructions from an instruction set, aggregating functionality of the first and second computer machine instructions to generate a third computer machine instruction, and adding the third computer machine instruction to the instruction set.

Hierarchical synthesis of computer machine instructions

Aspects relate to a method of generating a high-resolution image containing depth information of an object. In one aspect, the method includes downsampling a first reference image and a second reference image from a first resolution to a second resolution, wherein the first resolution is higher than the second resolution, and wherein the first reference image and the second reference image comprising a stereo image pair. The method further includes generating a depth map at the second resolution based on global minimization techniques, using the downsampled stereo image pair. The method also includes upsampling the depth map from the second resolution to the first resolution and using a guided filter to align contours of the upsampled depth map to contours of the first reference image.

Systems and methods for stereo depth estimation using global minimization and depth interpolation

Methods and apparatus for active depth sensing are disclosed. In some aspects, an imaging device may generate first depth information based on an active sensing technology, such as structured light. In some aspects, at least some of the first depth information may be missing or inaccurate, perhaps due to an extended range between the imaging device and a subject of the image. Additional range information may then be generated based on a zero order component of the structured light. The additional range information may then be used alone or combined with the first depth information and used to control one or more parameters of an imaging device, such as an exposure time and/or aperture.

Certain aspects relate to systems and techniques for performing local intensity equalization on images in a set of images exhibiting local intensity variations. For example, the local intensity equalization can be used to perform accurate region matching and alignment of the images. The images can be partitioned into regions of pixel blocks, for instance based on location, shape, and size of identified keypoints in the images. Regions depicting the same feature in the images can be equalized with respect to intensity. Region matching based on the keypoints in the intensity-equalized regions can be performed with accuracy even in images captured by asymmetric sensors or exhibiting spatially varying intensity.

Local adaptive histogram equalization

Systems and methods for correcting errors in a depth map generated by a structured light system are disclosed. In one aspect, a method includes dividing a depth map into segments and calculating a density distribution of the depth values for each segment. The method includes detecting error (or 'outlier') values by determining the depth values that fall outside of a range of depth values, the range of depth values representative of the highest density depth values for a given segment. The method includes detecting error values in the depth map as a whole based on the density distribution values for each segment.

Growing community of inventors

Carlsbad, CA, United States of America

Stephen Michael Verrall