Sony Corporation

Sony Group Corporation

Ying Lu

Yasutaka Hirasawa

Yuhi Kondo

Teppei Kurita

Shun Kaizu

Tuo Zhuang

Ayaka Nakatani

Legong Sun

Hideki Oyaizu

Akihiko Kaino

Hajime Mihara

A normal candidate information generation unit of an information processor generates normal candidate information for each pixel indicating, for example, a zenith angle, or an azimuth angle, or a zenith angle and an azimuth angle, on the basis of a polarization image in a plurality of polarization directions obtained by a polarization imaging unit. The in-plane pixel selection unit selects a plurality of pixels indicating the plane to be observed in the polarization image. A normal calculation unit calculates a normal of the plane to be observed on the basis of the normal candidate information of the pixels selected by the in-plane pixel selection unit.

Information processing apparatus and method for processing information

Provided is a polarization imaging unit that acquires a polarization image using a lens. In a parameter storage unit of an information processing unit, parameters related to a change in polarization state due to the lens which are estimated in advance from polarization state information acquired by imaging a light source in a predetermined polarization state using the lens and polarization state information indicating the polarization state of the light source, for example, change parameters indicating the change in polarization state due to the lens or correction parameters for correcting the change in polarization state due to the lens are stored. A polarization state correction unit of the information processing unit corrects the change in polarization state due to the lens, using the parameters stored in advance in the parameter storage unit.

Information processing apparatus, information processing method, and calibration apparatus

An unpolarized component image generating section generates unpolarized component image signals of specific and non-specific colors. A polarized component image generating section generates polarized component image signals of the specific and non-specific colors. A specific color polarization property detecting section detects polarization properties of the specific color using pixel signals of polarized or unpolarized pixels of the specific color having at least three polarization directions and pixel signals of polarized pixels having two polarization directions. An unpolarized color polarization property detecting section detects polarization properties of the non-specific color on the basis of the polarization properties of the specific color detected by the specific color polarization property detecting section, the pixel signals of the unpolarized pixels of the non-specific color, and the pixel signals of the polarized pixels of the non-specific color having fewer polarization directions than the polarized pixels of the specific color.

Image processing apparatus, image processing method, and solid-state imaging apparatus to detect polarization properties of each color

A third imaging unit including a pixel not having a polarization characteristic is interposed between a first imaging unit and a second imaging unit including a pixel having a polarization characteristic for each of a plurality of polarization directions. A depth map is generated from a viewpoint of the first imaging unit by matching processing using a first image generated by the first imaging unit and a second image generated by the second imaging unit. A normal map is generated on the basis of a polarization state of the first image. Integration processing of the depth map and the normal map is performed and a depth map with a high accuracy is generated. The depth map generated by the map integrating unit is converted into a map from a viewpoint of the third imaging unit, and an image free from deterioration can be generated.

Imaging device, image processing device and image processing method

A correction information generation sectionemits measurement light having uniform intensity onto a polarized image acquisition sectionacquiring a polarized image. Further, on the basis of a measured polarized image acquired from the polarized image acquisition section, the correction information generation sectiongenerates variation correction information for correcting sensitivity variations caused in the measured polarized image due to difference in polarization direction, and causes a correction information storage sectionto store the generated variation correction information. A correction processing sectionthen corrects the sensitivity variations caused in the polarized image acquired by the polarized image acquisition sectiondue to the difference in polarization direction by using the pre-generated variation correction information stored in the correction information storage section. Therefore, the polarized image outputted from the correction processing sectionbecomes a high-quality polarized image with, for example, its sensitivity variations corrected.

Image processing device, information generation device, and information generation method

An image processing device and a method that enable removal of reflection component light from images captured from various directions are provided. The image processing device includes: an image input unit that acquires at least three types of images formed with polarization images of different polarizing directions or non-polarization images; an angle input unit that outputs reflection angles that are the angles between the image capturing directions of the acquired images and the plane direction corresponding to the normal direction of the reflecting surface, to an intensity transmittance and intensity reflectance calculation unit; a calculation unit that calculates the respective intensity transmittances and the respective intensity reflectances of S waves and P waves, using the reflection angles; and a reflection component separation unit that removes the reflected light component from the images acquired by the image input unit, using the respective intensity transmittances and the respective intensity reflectances of the S waves and P waves. These processes enable removal of reflected light components from captured images having various reflection angles.

Image processing device and image processing method

A third imaging unit including a pixel not having a polarization characteristic is interposed between a first imaging unit and a second imaging unit including a pixel having a polarization characteristic for each of a plurality of polarization directions. A depth map is generated from a viewpoint of the first imaging unit by matching processing using a first image generated by the first imaging unit and a second image generated by the second imaging unit A normal map is generated on the basis of a polarization state of the first image. Integration processing of the depth map and the normal map is performed and a depth map with a high accuracy is generated. The depth map generated by the map integrating unit is converted into a map from a viewpoint of the third imaging unit, and an image free from deterioration can be generated.

An image pickup unithas a configuration in which non-polarizing pixels and polarizing pixels are disposed, the polarizing pixels being provided per angle in at least two polarization directions. A demosaicing unitgenerates a non-polarized image, and a polarization component image per polarization direction, from a captured image generated by the image pickup unit. A polarization information generating unitgenerates polarization information indicating the polarization characteristics of a subject included in the captured image, from the non-polarized image and the polarization component image generated by the demosaicing unit. As described above, the polarization information is generated with not only the polarization component image but also the highly-sensitive non-polarized image not having a decrease in the amount of light. Therefore, accurate polarization information can be acquired compared to a case where polarization information is generated on the basis of the polarization component image.

Image processing apparatus, image processing method, and image pickup device

A normal-line information generation blockgenerates normal-line information for a frame subject to detection. A data storage blockstores normal-line information and the like of a key frame. A motion amount detection blockdetects a motion amount of an imaging position of the frame subject to detection relative to an imaging position of the key frame on the basis of the normal-line information of the key frame stored in the data storage blockand the normal-line information of the frame subject to detection generated by the normal-line information generation block. Even if a positional difference of a same point is small or a luminance difference occurs between a taken image of the key frame and a taken image of a current frame, a motion amount of the imaging position of the frame subject to detection relative to the imaging position of the key frame can be accurately detected on the basis of normal-line information. Therefore, an observation position can be accurately detected.

Image processing apparatus and image processing method

The polarization imaging unit generates a polarized image including pixels for each of a plurality of polarization components. The demosaicing unit calculates a pixel signal for each polarization component by using the pixel signal of the target pixel of the polarized image and the pixel signal of the pixel for each of the identical polarization components located near the target pixel. In one example, a low frequency component is calculated for each polarization component using the pixel signal of the pixel located near the target pixel for each of the identical polarization components. In addition, component information indicating relationship between the low frequency component of the polarization component of the polarized image and the pixel signal of the target pixel is acquired. Furthermore, the pixel signal for each polarization component in the target pixel is calculated based on the low frequency component and the component information for each polarization component.

A polarized image acquisition sectionacquires a plurality of polarized images having different polarization directions. The polarized images show, for example, an input indicator for a user interface as a recognition target object. A normal line calculation sectioncalculates normal lines for individual pixels of the recognition target object in accordance with the polarized images acquired by the polarized image acquisition section. The normal lines represent information based on the three-dimensional shape of the recognition target object. A recognition sectionrecognizes the object by using the normal lines calculated by the normal line calculation section, determines, for example, the type, position, and posture of the input indicator, and outputs the result of determination as input information on the user interface. The object can be recognized easily and with high accuracy.

An imaging unithas a configuration in which an identical polarization pixel block made up of a plurality of pixels with an identical polarization direction is provided for each of a plurality of polarization directions and pixels of respective predetermined colors are provided in the identical polarization pixel block. A correction processing unitperforms correction processing such as white balance correction on a polarized image generated by the imaging unit. A polarized image processing unitseparates or extracts a reflection component using the polarized image after the correction processing. By using a polarized image of the separated or extracted reflection component, for example, it is possible to generate normal line information with high accuracy.

Image processing apparatus, image processing method, and image pickup element for separating or extracting reflection component

There is provided an image processing device including circuitry configured to generate an image of a subject under an illumination environment based on illumination information, from subject information which is associated with illumination of the subject and from the illumination information, wherein the illumination information is acquired on the basis of a virtual illumination body within a real space.

Growing community of inventors

Tokyo, Japan

Ying Lu