Pelican Imaging Corporation

Fotonation Limited

Fotonation Cayman Limited

Intrinsic Innovation LLC

Adeia Imaging LLC

Micron Technology Incorporated

Boston Polarimetrics, Inc.

Kip Peli P1 LP

Adela Imaging LLC

Aptina Imaging Corporation

Intel Corporation

Pelican Imaging Coporation

Kartik Venkataraman

Dan C Lelescu

Robert H Mullis

Amandeep S Jabbi

Gabriel Molina

Semyon Nisenzon

Jacques Duparre

Achuta Kadambi

Agastya Kalra

Florian Ciurea

Shane Ching-Feng Hu

Ankit K Jain

Andrew Kenneth John McMahon

Paul Gallagher

Vage Taamazyan

Priyam Chatterjee

Supreeth Krishna Rao

Jacques Duparré

Ramesh Raskar

Bedabrata Pain

Manohar B Srikanth

Shree Kumar Nayar

Ravi Ramamoorthi

Rishav Agarwal

Guy Michael Stoppi

Samuel Yang

Abhijit Ghosh

Pravin Rao

Robert Mullis

Cheng Lu

Bradley Craig Anderson Brown

Tim Poston

Anton Boykov

Yusong Huang

Junqing Chen

Semyon Nisenon

Andrew Kenneth John Mcmahon

Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints

Systems and methods for hybrid depth regularization in accordance with various embodiments of the invention are disclosed. In one embodiment of the invention, a depth sensing system comprises a plurality of cameras; a processor; and a memory containing an image processing application. The image processing application may direct the processor to obtain image data for a plurality of images from multiple viewpoints, the image data comprising a reference image and at least one alternate view image; generate a raw depth map using a first depth estimation process, and a confidence map; and generate a regularized depth map. The regularized depth map may be generated by computing a secondary depth map using a second different depth estimation process; and computing a composite depth map by selecting depth estimates from the raw depth map and the secondary depth map based on the confidence map.

Systems and methods for hybrid depth regularization

A computer-implemented method for computing a prediction on images of a scene includes: receiving one or more polarization raw frames of a scene, the polarization raw frames being captured with a polarizing filter at a different linear polarization angle; extracting one or more first tensors in one or more polarization representation spaces from the polarization raw frames; and computing a prediction regarding one or more optically challenging objects in the scene based on the one or more first tensors in the one or more polarization representation spaces.

Systems and methods for transparent object segmentation using polarization cues

A system for collecting data for training a computer vision model for shape estimation includes: an imaging system configured to capture one or more images; and a processing system including a processor and memory storing instructions that, when executed by the processor, cause the processor to: receive one or more input images from the imaging system; estimate a pose of an object depicted in the one or more images; render a shape estimate from a 3-D model of the object posed in accordance with the pose of the object; and generate a data point of a training dataset, the data point including one or more images based on the one or more input images and a label corresponding to the one or more images, the label including the shape estimate.

Systems and methods for generating and using visual datasets for training computer vision models

A data capture stage includes a frame at least partially surrounding a target object, a rotation device within the frame and configured to selectively rotate the target object, a plurality of cameras coupled to the frame and configured to capture images of the target object from different angles, a sensor coupled to the frame and configured to sense mapping data corresponding to the target object, and an augmentation data generator configured to control a rotation of the rotation device, to control operations of the plurality of cameras and the sensor, and to generate training data based on the images and the mapping data.

Systems and methods for training pose estimators in computer vision

A sensor system includes: a radar system configured to emit a radar beam and receive reflected radar signals from in a field of view of the radar system; a camera system including one or more cameras, at least one camera including a linear polarization filter in an optical axis of the camera, a field of view of the camera system overlapping the field of view of the radar system; and a processing system including a processor and memory, the memory storing instructions that, when executed by the processor, cause the processor to: receive radar data based on the reflected radar signals captured by the radar system; receive polarization raw frames captured by the camera system; and compute a track of a target in the field of view of the camera system and the field of view of the radar system based on the radar data and the polarization raw frames.

Sensor fusion between radar and optically polarized camera

Systems and methods in accordance with embodiments of the invention are disclosed that use super-resolution (SR) processes to use information from a plurality of low resolution (LR) images captured by an array camera to produce a synthesized higher resolution image. One embodiment includes obtaining input images using the plurality of imagers, using a microprocessor to determine an initial estimate of at least a portion of a high resolution image using a plurality of pixels from the input images, and using a microprocessor to determine a high resolution image that when mapped through the forward imaging transformation matches the input images to within at least one predetermined criterion using the initial estimate of at least a portion of the high resolution image. In addition, each forward imaging transformation corresponds to the manner in which each imager in the imaging array generate the input images, and the high resolution image synthesized by the microprocessor has a resolution that is greater than any of the input images.

Systems and methods for synthesizing high resolution images using images captured by an array of independently controllable imagers

Systems and method for an object from a plurality of objects are disclosed. An image of a scene containing the plurality of objects is obtained, and a segmentation map is generated for the objects in the scene. The shapes of the objects are determined based on the segmentation map. An end effector is adjusted in response to determining the shapes of the objects. The adjusting the end effector includes shaping the end effector according to at least one of the shapes of the objects. The plurality of objects is approached in response to the shaping of the end effector, and one of the plurality of objects is picked with the end effector.

Systems and methods for a vision guided end effector

A method for controlling a robotic system includes: capturing, by an imaging system, one or more images of a scene; computing, by a processing circuit including a processor and memory, one or more instance segmentation masks based on the one or more images, the one or more instance segmentation masks detecting one or more objects in the scene; computing, by the processing circuit, one or more pickability scores for the one or more objects; selecting, by the processing circuit, an object among the one or more objects based on the one or more pickability scores; computing, by the processing circuit, an object picking plan for the selected object; and outputting, by the processing circuit, the object picking plan to a controller configured to control an end effector of a robotic arm to pick the selected object.

Systems and methods for picking objects using 3-D geometry and segmentation

A polarized event camera system includes: an event camera having a field of view centered around an optical axis, the event camera including an image sensor including a plurality of pixels, each pixel of the event camera operating independently and asynchronously and being configured to generate change events based on changes in intensity of light received by the pixel; and a rotatable linear polarizer aligned with the optical axis of the event camera, the rotatable linear polarizer having a polarization axis, the polarization axis of the rotatable linear polarizer being rotatable about the optical axis of the event camera.

Polarization event cameras

A method of performing surface profilometry includes receiving one or more polarization raw frames of a printed layer of a physical object undergoing additive manufacturing, the one or more polarization raw frames being captured at different polarizations by one or more polarization cameras, extracting one or more polarization feature maps in one or more polarization representation spaces from the one or more polarization raw frames, obtaining a coarse layer depth map of the printed layer, generating one or more surface-normal images based on the coarse layer depth map and the one or more polarization feature maps, and generating a 3D reconstruction of the printed layer based on the one or more surface-normal images.

Systems and methods for surface normals sensing with polarization

Embodiments of the invention provide a camera array imaging architecture that computes depth maps for objects within a scene captured by the cameras, and use a near-field sub-array of cameras to compute depth to near-field objects and a far-field sub-array of cameras to compute depth to far-field objects. In particular, a baseline distance between cameras in the near-field subarray is less than a baseline distance between cameras in the far-field sub-array in order to increase the accuracy of the depth map. Some embodiments provide an illumination near-IR light source for use in computing depth maps.

Multi-baseline camera array system architectures for depth augmentation in VR/AR applications

A stereo camera array system includes: a first camera array at a first viewpoint including: a first camera configured to capture images in a first modality, the first modality being viewpoint-independent; and a second camera configured to capture images in a second modality different from the first modality; and a second camera array at a second viewpoint spaced apart along a first baseline from the first camera array at the first viewpoint, the second camera array including: a first camera configured to capture images in the first modality; and a second camera configured to capture images in the second modality.

Multi-modal and multi-spectral stereo camera arrays

A method for estimating a pose of an object includes: receiving, by a processor, an observed image depicting the object from a viewpoint; computing, by the processor, an instance segmentation map identifying a class of the object depicted in the observed image; loading, by the processor, a 3-D model corresponding to the class of the object; computing, by the processor, a rendered image of the 3-D model in accordance with an initial pose estimate of the object and the viewpoint of the observed image; computing, by the processor, a plurality of dense image-to-object correspondences between the observed image of the object and the 3-D model based on the observed image and the rendered image; and computing, by the processor, the pose of the object based on the dense image-to-object correspondences.

Systems and methods for using computer vision to pick up small objects

Systems and methods for depth estimation in accordance with embodiments of the invention are illustrated. One embodiment includes a method for estimating depth from images. The method includes steps for receiving a plurality of source images captured from a plurality of different viewpoints using a processing system configured by an image processing application, generating a target image from a target viewpoint that is different to the viewpoints of the plurality of source images based upon a set of generative model parameters using the processing system configured by the image processing application, and identifying depth information of at least one output image based on the predicted target image using the processing system configured by the image processing application.

Systems and methods for depth estimation using generative models

Growing community of inventors

San Jose, CA, United States of America

Kartik Venkataraman