Intel Corporation

Interval Research Corporation

Vulcan Patents LLC

Tyzx, Inc.

Interval Licensing LLC

John Iselin Woodfill

Gaile Gordon

Henry Harlyn Baker

Pierre St Hilaire

Michael Harville

Etienne Gregoire Grossmann

Brian Von Herzen

Robert Dale Alkire

Trevor J Darrell

Nicholas R Kalayjian

Harlyn Baker

Ronald John Buck

Michele Mae Covell

Paul Winer

Anders Grunnet-Jepsen

John N Sweetser

Akihiro Takagi

Steve John Clohset

Ali Rahimi

David Walter Jurasek

Michael Hongmai Lin

Terrence Lee Brown

Brett Miller

A light pattern projector with a pattern mask to spatially modulate an intensity of a wideband illumination source, such as an LED, and a projector lens to reimage the spatially modulated emission onto regions of a scene that is to be captured with an image sensor. The projector lens may comprise a microlens array (MLA) including a first lenslet to reimage the spatially modulated emission onto a first portion of a scene, and a second lenslet to reimage the spatially modulated emission onto a first portion of a scene. The MLA may have a fly's eye architecture with convex curvature over a diameter of the projector lens in addition to the lenslet curvature. The pattern mask may be an amplitude mask comprising a mask pattern of high and low amplitude transmittance regions. In the alternative, the pattern mask may be a phase mask, such as a refractive or diffractive mask.

LED pattern projector for 3D camera platforms

Techniques are provided for range image generation. An input image and a reference image are used to compute raw and optimized disparity-space correlation disparity results for each pixel in the reference image. A bilateral filter is applied to both the raw and optimized disparity space correlation disparity results within a neighborhood of a given pixel. For each pixel in the neighborhood, a mean of sub-pixel disparity estimates includes the raw disparity-space correlation disparity result, and another mean of sub-pixel disparity estimates is computed using optimized disparity-space correlation disparity results. The output of the bilateral filter is a weighted sum of the means, with the weights being proportional to the ratio of valid, non-optimized sub-pixel correlation disparity results to optimized sub-pixel correlation disparity results, thereby favoring non-optimized estimates. A sub-pixel range image is generated based at least in part on the sub-pixel disparity estimate for each pixel in the reference image.

Range image generation

Methods, systems, and computer program products for determining, without interrupting a video processing pipeline, whether currently used warping parameters are appropriate for a multi-camera system. New warping parameters may be determined, where these parameters cause the images rectified with these new parameters to have the desired geometric properties. This determination may be made without interrupting the video processing pipeline. Warping parameters may be determined, for example, such that the epipolar lines of the two cameras are horizontal and properly aligned, as defined below, and the metric 3D position of a point observed by both cameras can be determined and is accurate.

Systems, methods, and computer program products for runtime adjustment of image warping parameters in a multi-camera system

Methods, systems, and computer program products to obtain a color image from a color filter array, such as, for example, a color filter array comprising a Bayer pattern. A method of image processing includes, for each pixel location (i,j) and for each color channel in a color image, determining a warped location (i',j&#x2032;) in a color filter array, and determining a color value of the color channel at location (i&#x2032;,j&#x2032;) in the color filter array. The method may further include storing the determined color value. The determining of the color value may include interpolating the color value of the color channel at location (i&#x2032;,j&#x2032;) in the color filter array. In this manner, a single interpolation operation set may be needed and an intermediate image may be avoided, saving memory, reducing processing time, minimizing artifacts, and reducing cost.

Systems, methods, and computer program products for compound image demosaicing and warping

Systems and methods to determine a disparity map using row causal scanline optimization stereo matching are presented. A method includes, for each corresponding pixel P between a pair of input stereo images, and for each considered disparity, determining a basic match cost and a match cost for each of a set of given orientations including an east orientation and one or more other orientations, determining an overall match cost for each pixel at each considered disparity based on a sum of the determined match costs for all considered orientations for each pixel and disparity pair, and determining a resulting disparity for each pixel based on a minimum of the determined overall match costs, where a subset of the determined resulting disparities becomes available prior to completion of the input images being read in, and where the resulting disparities for all pixels are determined in a single pass through the input images.

Systems and methods for row causal scan-order optimization stereo matching

Methods, systems, and computer program products to warp a depth map into alignment with an image, where the image sensor (e.g., camera) responsible for the image and depth sensor responsible for an original depth map are separated in space. In an embodiment, the warping of the depth map may be started before the original depth map has been completely read. Moreover, data from the warped depth map may be made available to an application before the entire warped depth map has been completely generated. Such a method and system may improve the speed of the overall process and/or reduce memory requirements.

Systems, methods, and computer program products for low-latency warping of a depth map

A system for computing one or more calibration parameters of a camera is disclosed. The system includes a processor and a memory. The processor is configured to provide a first object either marked with or displaying three or more fiducial points. The fiducial points have known 3D positions in a first object reference frame. The processor is further configured to provide a second object either marked with or displaying three or more fiducial points. The fiducial points had known 3D positions in a second object reference frame. The processor is further configured to place the first object and the second object in a fixed position such that the fiducial point positions of the first and second objects are non-planar. The processor is further configured to compute one or more calibration parameters of the second camera using computations based on images taken of the fiducials.

Camera calibration using an easily produced 3D calibration pattern

A system for determining one or more camera calibration parameters is disclosed. The system comprises a processor and a memory. The processor is configured to: a) provide a first pattern for display on a display screen; b) receive a first image from a camera viewing the display screen; c) provide a second pattern for display on the display screen; and d) receive a second image from the camera viewing the display screen. The relative position of the display screen and the camera are the same as when the first image was captured using the camera. The processor is further configured to determine an image location which is a projection of a known physical location on the display screen by using at least in part a first feature identified in the first image and a second feature identified in the second image and determine one or more calibration parameters based at least in part on the determined image location. The memory is coupled to the processor and configured to provide the processor with instructions.

Display screen for camera calibration

A system is disclosed. An input interface is configured to receive pixel data from two or more images. A pixel handling processor disposed on the substrate is configured to convert the pixel data into depth and intensity pixel data. In some embodiments, a foreground detector processor disposed on the substrate is configured to classify pixels as background or not background. In some embodiments, a projection generator disposed on the substrate is configured to generate a projection in space of the depth and intensity pixel data.

Integrated image processor

A precision motion platform carrying an imaging device under a large-field-coverage lens enables capture of high resolution imagery over the full field in an instantaneous telephoto mode and wide-angle coverage through temporal integration. The device permits automated tracking and scanning without movement of a camera body or lens. Coupled use of two or more devices enables automated range computation without the need for subsequent epipolar rectification. The imager motion enables sample integration for resolution enhancement. The control methods for imager positioning enable decreasing the blur caused by both the motion of the moving imager or the motion of an object's image that the imager is intended to capture.

Moving imager camera for track and range capture

A system for generating disparity results comprises an interface, a first memory, a second memory, and a processor. The interface is for receiving a first element of a first set of image data and a first element of a second set of image data. The first memory is for storing the first element of the first set of image data. The second memory is for storing the first element of the second set of image data. The processor is for generating a disparity result for a first element before all elements of the first data set and the second data set have been received. The disparity result is generated using a low latency image processing system that processes a plurality of elements of the first set of image data and a plurality of elements of the second set of image data.

Generation of a disparity result with low latency

A powerful, scaleable, and reconfigurable image processing system and method of processing data therein is described. This general purpose, reconfigurable engine with toroidal topology, distributed memory, and wide bandwidth I/O are capable of solving real applications at real-time speeds. The reconfigurable image processing system can be optimized to efficiently perform specialized computations, such as real-time video and audio processing. This reconfigurable image processing system provides high performance via high computational density, high memory bandwidth, and high I/O bandwidth. Generally, the reconfigurable image processing system and its control structure include a homogeneous array of 16 field programmable gate arrays (FPGA) andstatic random access memories (SRAM) arranged in a partial torus configuration. The reconfigurable image processing system also includes a PCI bus interface chip, a clock control chip, and a datapath chip. It can be implemented in a single board. It receives data from its external environment, computes correspondence, and uses the results of the correspondence computations for various post-processing industrial applications. The reconfigurable image processing system determines correspondence by using non-parametric local transforms followed by correlation. These non-parametric local transforms include the census and rank transforms. Other embodiments involve a combination of correspondence, rectification, a left-right consistency check, and the application of an interest operator.

A system for distance calculation is disclosed. The system includes an illuminator unit, one or more camera units, and a distance processor. The illuminator unit illuminates a scene in a target area using a textured pattern creator and wherein the textured pattern creator includes a diffractive optical element. The one or more camera units captures two or more images of the target area from two or more physical locations. A textured pattern illumination is visible in each of the two or more images of the target area. The images are used to calculate distances to one or more points in the scene in the target area.

Growing community of inventors

San Francisco, CA, United States of America

John Iselin Woodfill