Nikon Corporation

Kenichi Ishiga

Kiyoshige Shibazaki

Muneki Hamashima

Susumu Mori

Akihiko Utsugi

Ken Utagawa

Satoshi Ejima

Sadami Okada

Zhe-Hong Chen

Hideaki Matsuda

Hideki Matsuda

Mamoru Iesaka

Junya Hagiwara

Yusuke Takanashi

Fumiki Nakamura

To produce both 2D image data and color parallax image data from image data output from a single-plate image sensor, the resolutions of the 2D image data and parallax image data may be both degraded. The image sensor has a primitive lattice that is a group of pixels including (i) at least four types of parallax pixels formed by photoelectric converter elements each of which is associated with one of combinations of two different types of aperture masks and two different types of color filters and (ii) no-parallax pixels configured to guide an incident luminous flux to photoelectric converter elements without limitation. In the group of pixels, the no-parallax pixels are more than the parallax pixels.

Image sensor, image-capturing apparatus and image-capturing system

Reception of the invisible light wavelength band in an incident luminous flux has not been considered. In view of this, an image sensor is provided, the image sensor including: a visible parallax pixel that is associated with a visible range light-receiving photoelectric conversion pixel having any of a plurality of types of aperture mask each including an aperture that is positioned to allow passage, to a photoelectric converting element, of a mutually different partial luminous flux of an incident luminous flux in a visible light wavelength band; and an invisible parallax pixel that is associated with an invisible range light-receiving photoelectric conversion pixel having any of a plurality of types of aperture mask each including an aperture positioned to allow passage, to the photoelectric converting element, of a mutually different partial luminous flux in an incident luminous flux in an invisible light wavelength band.

Image sensor, imaging device and image processing device that generate parallax images in both visible and invisible wavelength bands

A technique is disclosed to obtain first image data including a first pixel value of a first pixel having a no-parallax opening corresponding to a reference direction, a second pixel value of a second pixel having a parallax opening that causes parallax in one direction and a third pixel value of a third pixel having a parallax opening that causes parallax in the opposite direction, to calculate, for a pixel position of the second pixel, a fourth pixel value by averaging the second pixel value at the pixel position and a hypothetical third pixel value and to calculate, for a pixel position of the third pixel, a fifth pixel value by averaging the third pixel value at the pixel position and a hypothetical second pixel value, and to use the first pixel value, the fourth pixel value and the fifth pixel value to produce 2D second image data.

Imaging processing apparatus, image-capturing apparatus, and storage medium having image processing program stored thereon

An image processing device including means for: inputting two parallax images composed of images of first and second viewpoint directions; generating first and second smoothed images by applying, respectively to the images of the first and second viewpoint directions, a smoothing filter in a direction joining the first and second viewpoints; generating a reference image formed by a value based on a product of values of the first and second smoothed images; generating gain distribution data by dividing a value of the reference image by the value of the first smoothed image; performing gain correction of each pixel on the image of the first viewpoint direction, based on the gain distribution data; and outputting the gain corrected image of the first viewpoint direction as a parallax image, the smoothing filter having a filter diameter approximately equal to a parallax amount between the images of the first and second viewpoint directions.

Image processing device, imaging apparatus, and computer-readable medium having stored thereon an image processing program

Image sensor, image-capturing apparatus and image capturing system

An image processing apparatus, comprising: a receiving unit receiving a first image and image-capturing information associated with the first image; a determining unit determining, based on the image-capturing information, whether or not the first image is an image of a subject image in a view point direction corresponding to the aperture mask, using an output signal of an image sensor that includes a pixel array including first parallax pixels having the aperture mask with asymmetric directional properties in the long axis and short axis with respect to incident light flux of a single optical system and positioned to permit a portion of the light flux to pass; and an outputting unit outputting a corrected first image by carrying out a correction process having different frequency properties between the long axis direction and the short axis direction on the subject image in an unfocused region of the first image.

Image processing apparatus, image-capturing apparatus and image processing apparatus control program using parallax image data having asymmetric directional properties

An image processing apparatus is provided, that is configured to: extract a first pixel value corresponding to a first viewpoint that is one of a plurality of viewpoints to capture a subject image, at a target pixel position from image data having the first pixel value; extract second and third luminance values corresponding to second and third viewpoints that are different from the first viewpoint, at the target pixel position from luminance image data having the second and third luminance values; and calculate at least any of second and third pixel values of the second and third viewpoints such that a relational expression between the second or third pixel value and the first pixel value extracted by the pixel value extracting unit remains correlated with a relational expression defined by the second and third luminance values.

Image processing device, imaging device, and image processing program

An image sensor captures a subject image of a partial luminous flux passed through different regions from among a total luminous flux through one optical system. The image sensor is configured from a pixel arrangement in which a plurality of each of at least three types of pixels are arranged, including non-parallax pixels which comprise an aperture mask that produces a viewpoint in a reference direction, first parallax pixels which comprise an aperture mask that produces a viewpoint in a first direction different from the reference direction and second parallax pixels which comprise an aperture mask that produces a viewpoint in a second direction different from the reference direction, wherein each of the aperture mask of the first parallax pixels and the aperture mask of the second parallax pixels has a width of a greater-than-half-aperture aperture area in the direction of change of the viewpoint.

Image sensor, imaging apparatus, and image processing device

A technique is disclosed to calculate a new pixel value of a particular color of second-viewpoint color image data in such a manner that a difference between the new pixel value of the particular color of the second viewpoint and the pixel value of the particular color of the first-viewpoint color image data maintains correlation with a sum of a difference between the pixel value of a first color component of the second-viewpoint color image data and the pixel value of the first color component of third-viewpoint color image data and a difference between the pixel value of a second color component of the second-viewpoint color image data and the pixel value of the second color component of the third-viewpoint color image data.

Image processing apparatus, image-capturing apparatus, and storage medium having image processing program stored thereon

An image processing apparatus includes a receiving unit configured to receive at least two parallax images that are obtained from a subject image captured via a single optical system, where the at least two parallax images include an image in a first viewpoint direction and an image in a second viewpoint direction, an average calculating unit configured to calculate, for each pixel, an arithmetic average and a geometric average between the image in the first viewpoint direction and the image in the second viewpoint direction, a ratio calculating unit configured to calculate, for each pixel, a ratio of the arithmetic average to the geometric average, and a disparity calculating unit configured to calculate, on a pixel-by-pixel basis, a disparity between the image in the first viewpoint direction and the image in the second viewpoint direction based on the ratio.

Image processing apparatus, image-capturing apparatus and image processing method

An imaging apparatus including an image generating section that generates first parallax image data based on the output of the first pixel and second parallax image data based on the output of the second pixel. When the imaging element captures an image of an object point located in an unfocused region on the optical axis, a pixel value of a center pixel corresponding to the optical axis in the first parallax image data is greater than or equal to 50% of a pixel value of a peak pixel in the first parallax image data, and a pixel value of a center pixel corresponding to the optical axis in the second parallax image data is greater than or equal to 50% of a pixel value of a peak pixel in the second parallax image data.

Imaging apparatus for generating parallax image data

The present invention includes the steps of: generating a temporary left parallax image using the pixel value of the left parallax pixels of the first image for each pixel; generating a temporary right parallax image using the pixel value of the right parallax pixels of the first image for each pixel; generating a standard view point image using the pixel values of at least the non-parallax pixels of the first image for each pixel; generating an edge enhanced standard view point image, by carrying out an edge enhancement process on the standard view point image; and generating a left direction view point image and a right direction view point image based on the edge enhanced standard view point image and the temporary left parallax image and the temporary right parallax image for each pixel.

Image processing method, image processing apparatus, image-capturing apparatus, and image processing program

An image processing device that corrects a pixel value in image data, includes: an input unit that inputs image data having a plurality of color components and including a plurality of pixels each holding one color component; a calculation unit that calculates a maximum value among pixel values of a plurality of same-color neighboring pixels each holding a color component identical to a color component held in a processing target pixel selected from the pixels included in the image data and pixel values of a plurality of different-color neighboring pixels each holding a color component different from the color component held in the processing target pixel, which take up positions closer to the processing target pixel than the plurality of same-color neighboring pixels; and a correction unit that compares a pixel value of the processing target pixel with the maximum value and replaces the pixel value of the processing target pixel with the maximum value if the pixel value of the processing target pixel is greater than the maximum value.

Maximum and minimum filter in mosaic image

An image processing method for removing a noise component contained in an original image, includes: separating an original image into a temporary noise-free image and a temporary noise component; extracting an edge component in the temporary noise-free image by executing edge detection on the temporary noise-free image; determining a fine edge component in the original image contained in the temporary noise component based upon a level of the extracted edge component; extracting an actual noise component by excluding the fine edge component from the temporary noise component; and removing noise from the original image based upon the extracted actual noise component.

Image processing for removing a noise component contained in an image

An image processing device that converts a first image captured via an optical system in which at least one of a plurality of color components is missing in one pixel and MTF characteristics are different between a reference color component and at least one missing color component at an imaging plane, into a second image in which MTF characteristics are matched, includes: an image creation unit that acquires information concerning differences in MTF characteristics between the missing color component and the reference color component in a pixel having the missing color component of the first image and creates the second image by using the acquired information.

Growing community of inventors

Yokohama, Japan

Kenichi Ishiga