Sony Corporation

Nikon Corporation

Sony Electronics Inc

Gazi Ali

Radka Tezaur

Ming-Chang Liu

Florian Ciurea

Pingshan Li

Li Hong

Akira Matsui

Alexander Berestov

Takami Mizukura

Chuen-Chien Lee

Mark K Takita

Hideki Nabesako

Tetsuji Kamata

Ximin Zhang

Nobuyuki Matsushita

Kenichi Nishio

Kensuke Miyagi

Deblurring a blurry image () includes the steps of (i) computing a spatial mask (); (ii) computing a modified blurry image () using the blurry image () and the spatial mask (); and (iii) computing a latent sharp image () using the modified blurry image () and a point spread function (). Additionally, the image () can be analyzed to identify areas of the image () that are suitable for point spread function estimation. Moreover, a region point spread function () can be analyzed to classify the point spread function(s) as representing (i) motion blur, (ii) defocus blur, or (iii) mixed motion blur and defocus blur. A point spread function () can also be estimated.

Algorithm and device for image processing

Deblurring a blurry image () includes the steps of (i) computing a spatial mask (); (ii) computing a modified blurry image () using the blurry image () and the spatial mask (); and (iii) computing a latent sharp image () using the modified blurry image () and a point spread function (). Additionally, the image () to can be analyzed to identify areas of the image () that are suitable for point spread function estimation. Moreover, a region point spread function () can be analyzed to classify the point spread function(s) as representing (i) motion blur, (ii) defocus blur, or (iii) mixed motion blur and defocus blur. A point spread function () can also be estimated.

A group point spread function () for a blurry image () can be determined by dividing the blurry image () into a plurality of image regions (), estimating a region point spread function () for at least two of the image regions (); and utilizing at least two of the region point spread functions () to determine the group point spread function (). The group point spread function () can be determined by the decomposition of the estimated region point spread functions () into some basis function, and subsequently determining a representative coefficient from the basis functions of the region point spread functions () to generate the group point spread function ().

Globally dominant point spread function estimation

A method for classifying a test image () includes the steps of building a classifier (), and classifying the test image () with the classifier (). After the test image () is classified, the test image () can be subsequently processed (e.g. deblurred) with improved accuracy. The classifier () can classify and distinguish between PSF features associated with motion blurred images, and PSF features associate with defocus blurred images. The classifier () can be built using a plurality of training images (), and extracting one or more training features from each the training images (). The PSF features can include image moments of the point spread function, and/or geometric features of the point spread function.

Point spread function classification using structural properties

A Depth Map (DM) is able to be utilized for many parameter settings involving cameras, camcorders and other devices. Setting parameters on the imaging device includes zoom setting, aperture setting and shutter speed setting.

Automatic setting of zoom, aperture and shutter speed based on scene depth map

A system and method for supporting a depth estimation procedure by utilizing an adaptive kernel includes a capture subsystem for capturing images of a photographic target. The capture subsystem includes an aperture that is adjustable for admitting reflected light from the photographic target to a sensor device. An adaptive kernel is designed in a kernel design procedure based upon symmetry characteristics of the aperture. The adaptive kernel may be designed in either a frequency-domain kernel design procedure or in a spatial-domain kernel design procedure. A depth estimator utilizes the adaptive kernel for performing the depth estimation procedure.

System and method for performing depth estimation by utilizing an adaptive kernel

A system and method for generating robust depth maps includes a depth estimator that creates a depth map pyramid structure that includes a plurality of depth map levels that each have different resolution characteristics. In one embodiment, the depth map levels include a fine-scale depth map, a medium-scale depth map, and a coarse scale depth map. The depth estimator evaluates depth values from the fine-scale depth map by utilizing fine-scale confidence features, and evaluates depth values from the medium-scale depth map and the coarse-scale depth map by utilizing coarse-scale confidence features. The depth estimator then fuses optimal depth values from the different depth map levels into an optimal depth map.

System and method for generating robust depth maps utilizing a multi-resolution procedure

Method and apparatus for performing visual effects rendering on an image are described. User specifications are processed for at least one visual effect for the image. Scene analysis information associated with the image may be accessed, wherein the scene analysis information is based on the user specifications and indicates at least one pixel of the image for coupling with the at least one visual effect. The at least one pixel is accentuated using depth map information, wherein dimensions for the at least one pixel are determined in response to corresponding depth values of the at least one pixel. An output image is generated having at least one visual effect.

Method and apparatus for performing user inspired visual effects rendering on an image

A method and apparatus for generating a dense depth map. In one embodiment, the method includes applying a joint bilateral filter to a first depth map to generate a second depth map, where at least one filter weight of the joint bilateral filter is adapted based upon content of an image represented by the first depth map, and the second depth map has a higher resolution than the first depth map.

Method and apparatus for generating a dense depth map using an adaptive joint bilateral filter

A method and apparatus for performing a blur rendering process on an image is disclosed. In one embodiment, the method of performing a blur rendering process includes accessing a filtered image and depth map information, determining a plurality of blending coefficients for computing a weighted sum for the image and filtered image, wherein the plurality of blending coefficients define a substantially smooth transition from at least one first depth class to at least one second depth class and a substantially sharp transition from the at least one second depth class and the at least one first depth class, wherein the at least one first depth class and the at least one second depth class form at least a portion of a plurality of depth classes and combining the image and the filtered image into a resulting image using the plurality of coefficients.

Method and apparatus for performing a blur rendering process on an image

A method and apparatus of depth of field rendering which simulates larger apertures for images captured at a smaller aperture. The depth of field rendering provides selective simulation of out-of-focus effects which are attainable with cameras having a larger aperture when capturing images at a smaller aperture. A blur function model is created based on the relationship between the blur change and the aperture change. This model is used to determine the blur difference which would arise between two images taken at two different apertures. Then the out-of-focus effect is generated by blurring the image in a rendering process based on the blur difference.

Blur function modeling for depth of field rendering

A method and apparatus for generating a super-resolution image are provided. The method may include obtaining a first set of RAW data representing a first image captured at a first resolution and obtaining, from the first set of RAW data, at least one first sample of data associated with the first image. The method may also include obtaining a second set of RAW data representing a second image captured at the first resolution, and performing image registration as a function of the first set of RAW data and the second set of RAW data so as to obtain at least one second sample of data associated with the second image. The first set of RAW data is used as a reference for the second set of RAW data. The method further includes combining the at least one first sample of data with at least one second sample of data to form a collection of samples, and interpolating the collection of samples to form the super-resolution image.

Method and apparatus for forming super resolution images from raw data representative of color filter array images

A method and apparatus for forming a demosaiced image from a color-filter-array ('CFA') image is provided. The CFA image comprises a first set of pixels colored according to a first (e.g., a green) color channel, a second set of pixels colored according to a second (e.g., a red) color channel and a third set of pixels colored according to a third (e.g., blue) color channel. The method may include obtaining an orientation map, which includes, for each pixel of the color-filter-array image, an indicator of orientation of an edge bounding such pixel. The method may further include interpolating the first color channel at the second and third sets of pixels as a function of the orientation map so as to form a fourth set of pixels. The method may also include interpolating the second color channel at the first and third sets of pixels as a function of the orientation map and the fourth set of pixels; and interpolating the third color channel at the first and second sets of pixels as a function of the orientation map and the fourth set of pixels.

Orientation-based approach for forming a demosaiced image, and for color correcting and zooming the demosaiced image

A method of image color tone correction is disclosed, which may include the steps of (a) providing a reference image and one or more source images, wherein all images are comprised of pixels of a scene; (b) correcting a designated portion of one of the source images based on the reference image to create a color tone corrected image; and (c) outputting the color tone corrected image to a computer readable medium. The designated portion may be any subset of one of the source images, and may be a simple cropping, interlace, or other form of selection, such as areas of interest in the scene. Additionally, the designated portion may be used for determining a set of coefficients for overall color image correction of the complement of the source images. This method brings the increased tonal and saturation qualities of still images to those in motion.

Growing community of inventors

Santa Clara, CA, United States of America

Gazi Ali