Sony Corporation

Pelican Imaging Corporation

Fotonation Limited

Fotonation Cayman Limited

Apple Inc.

Sony Electronics Inc

Adeia Imaging LLC

Florian Ciurea

Dan C Lelescu

Kartik Venkataraman

Gabriel Molina

Kensuke Miyagi

Pingshan Li

Alexander Berestov

Andrew Kenneth John McMahon

Semyon Nisenzon

Paul Gallagher

Ankit K Jain

Gazi Ali

Priyam Chatterjee

Tomonori Shuda

Jianping Zhou

Chuen-Chien Lee

Hideki Nabesako

Ahmed Tashrif Kamal

Paul M Hubel

Nobuyuki Matsushita

Naoya Katoh

Kenichi Nishio

Sebastien X Beysserie

Tatsuya Deguchi

Michael Wish Tao

Geoffrey T Anneheim

Zejing Wang

Julio Cesar Hernandez Zaragoza

Feng Li

Mamoru Sugiura

Stephane S Ben Soussan

Saeed Izadi

Eric Bujold

Stephane Simon Rene Ben Soussan

Volker Roelke

Benjamin A Darling

Mark Norman Lester Jouppi

Masato Watanabe

Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints

Devices, methods, and non-transitory computer readable media are disclosed herein to repair or mitigate the appearance of unwanted reflection artifacts in captured video image streams. These unwanted reflection artifacts often present themselves as brightly-colored spots, circles, rings, or halos that reflect the shape of a bright light source in the captured image. These artifacts, also referred to herein as 'ghosts' or “green ghosts” (due to often having a greenish tint), are typically located in regions of the captured images where there is not actually a bright light source located in the image. In fact, such unwanted reflection artifacts often present themselves on the image sensor across the principal point of the lens from where the actual bright light source in the captured image is located. Such devices, methods and computer readable media may be configured to detect, track, and repair such unwanted reflection artifacts in an intelligent and efficient fashion.

Efficient unwanted reflection artifact mitigation in videos for embedded devices

Devices, methods, and non-transitory program storage devices are disclosed to provide techniques for distortion compensation in images that are subject to video image stabilization (VIS), e.g., electronic image stabilization (EIS). In some embodiments, the techniques comprise: obtaining a first image captured by a first image capture device; determining a first set of image parameters configured to apply a first distortion operation (e.g., an un-distortion operation) to the first image; determining a second set of image parameters configured to apply a first stabilization operation to a first version of the first image that has already had the first distortion operation applied; determining a third set of image parameters configured to apply a second distortion operation (e.g., an idealized, field of view-preserving, re-distortion operation) to the first image; and applying the determined first, second, and third sets of image parameters to the first image to generate a distortion-compensated and stabilized output image.

Distortion compensation in video image stabilization

Systems and methods for dynamically calibrating an array camera to accommodate variations in geometry that can occur throughout its operational life are disclosed. The dynamic calibration processes can include acquiring a set of images of a scene and identifying corresponding features within the images. Geometric calibration data can be used to rectify the images and determine residual vectors for the geometric calibration data at locations where corresponding features are observed. The residual vectors can then be used to determine updated geometric calibration data for the camera array. In several embodiments, the residual vectors are used to generate a residual vector calibration data field that updates the geometric calibration data. In many embodiments, the residual vectors are used to select a set of geometric calibration from amongst a number of different sets of geometric calibration data that is the best fit for the current geometry of the camera array.

Systems and methods for dynamic calibration of array cameras

Electronic devices, computer readable storage media, and related methods are disclosed herein that are configured to stitch together images captured by multiple image capture devices of an image capture system. In particular, various techniques are employed to intelligently extend (and, optionally, smooth) the correspondence mapping between first and second images captured by image capture devices having different fields of view, e.g., fields of view that are at least partially overlapping and at least partially non-overlapping. The techniques may also include determining a 'transitional' correspondence in a transitional region between the overlapping and non-overlapping regions of the fields of view, as well as performing one or more appearance correction operations to account for the different properties of the different image capture devices used to capture the first and second images. The techniques described herein may be employed to produce enhanced output images in either the still image or the video context.

Field of view extension in a stereo camera system

System and methods for measuring depth using an array of independently controllable cameras

An electronic device comprises circuitry implementing a depth map enhancer. The depth map enhancer obtains an initial depth map corresponding to a scene and an image of the scene. The depth map enhancer generates a refined depth map corresponding to the scene using an optimizer, the initial depth map and the image. The refined depth map includes estimated depth indicators corresponding to at least a first depth-information region, identified based at least in part on a first criterion, of the initial depth map. Input based on the refined depth map is provided to an image processing application.

Efficient scene depth map enhancement for low power devices

Systems with an array camera augmented with a conventional camera in accordance with embodiments of the invention are disclosed. In some embodiments, the array camera is used to capture a first set of image data of a scene and a conventional camera is used to capture a second set of image data for the scene. An object of interest is identified in the first set of image data. A first depth measurement for the object of interest is determined and compared to a predetermined threshold. If the first depth measurement is above the threshold, a second set of image data captured using the conventional camera is obtained. The object of interest is identified in the second set of image data and a second depth measurement for the object of interest is determined using at least a portion of the first set of image data and at least a portion of the second set of image data.

Autofocus system for a conventional camera that uses depth information from an array camera

Systems and methods for performing photometric normalization in an array camera in accordance with embodiments of this invention are disclosed. The image data of scene from a reference imaging component and alternate imaging components is received. The image data from each of the alternate imaging components is then translated to so that pixel information in the image data of each alternate imaging component corresponds to pixel information in the image data of the reference component. The shifted image data of each alternate imaging component is compared to the image data of the reference imaging component to determine gain and offset parameters for each alternate imaging component. The gain and offset parameters of each alternate imaging component is then applied to the image data of the associate imaging to generate corrected image data for each of the alternate imaging components.

Systems and methods for photometric normalization in array cameras

Systems and methods for measuring scene information while capturing images using array cameras in accordance with embodiments of the invention are disclosed. In one embodiment, a method of measuring scene information while capturing an image using an array camera includes defining at least two subsets of active cameras, configuring the active cameras using image capture settings, capturing image data using the active cameras, synthesizing at least one image using image data captured by a first subset of active cameras, measuring scene information using image data captured by a second subset of active cameras, and determining whether the image capture settings satisfy at least one predetermined criterion for at least one image capture parameter using the measured scene information, where new image capture settings are determined and utilized to configure the active cameras upon a determination that the image capture settings do not satisfy the at least one predetermined criterion.

Systems and methods for measuring scene information while capturing images using array cameras

Growing community of inventors

San Jose, CA, United States of America

Florian Ciurea