Sony Corporation

Sony Interactive Entertainment Inc.

Ayaka Nakatani

Yasutaka Hirasawa

Yuhi Kondo

Ying Lu

Akio Ohba

Hiroyuki Segawa

Tetsugo Inada

Takayuki Shinohara

Akihiko Kaino

Hidehiko Morisada

Yoshihiro Myokan

Estimation data indicating a foreground region is generated with high precision. A learning apparatus includes an input-image acquisition section that acquires a combination of a first input image representing a background and a foreground and a second input image representing the background in a mode different from that in the first input image, and a learning section that includes an estimation section to generate estimation data indicating the foreground region in the first input image in response to input of the first input image and the second input image, and that conducts learning at the estimation section on the basis of a given teacher image and the estimation data that is generated when the first input image and the second input image are inputted.

Learning apparatus, foreground region estimation apparatus, learning method, foreground region estimation method, and program

A third imaging unit including a pixel not having a polarization characteristic is interposed between a first imaging unit and a second imaging unit including a pixel having a polarization characteristic for each of a plurality of polarization directions. A depth map is generated from a viewpoint of the first imaging unit by matching processing using a first image generated by the first imaging unit and a second image generated by the second imaging unit. A normal map is generated on the basis of a polarization state of the first image. Integration processing of the depth map and the normal map is performed and a depth map with a high accuracy is generated. The depth map generated by the map integrating unit is converted into a map from a viewpoint of the third imaging unit, and an image free from deterioration can be generated.

Imaging device, image processing device and image processing method

A third imaging unit including a pixel not having a polarization characteristic is interposed between a first imaging unit and a second imaging unit including a pixel having a polarization characteristic for each of a plurality of polarization directions. A depth map is generated from a viewpoint of the first imaging unit by matching processing using a first image generated by the first imaging unit and a second image generated by the second imaging unit A normal map is generated on the basis of a polarization state of the first image. Integration processing of the depth map and the normal map is performed and a depth map with a high accuracy is generated. The depth map generated by the map integrating unit is converted into a map from a viewpoint of the third imaging unit, and an image free from deterioration can be generated.

A normal-line information generation blockgenerates normal-line information for a frame subject to detection. A data storage blockstores normal-line information and the like of a key frame. A motion amount detection blockdetects a motion amount of an imaging position of the frame subject to detection relative to an imaging position of the key frame on the basis of the normal-line information of the key frame stored in the data storage blockand the normal-line information of the frame subject to detection generated by the normal-line information generation block. Even if a positional difference of a same point is small or a luminance difference occurs between a taken image of the key frame and a taken image of a current frame, a motion amount of the imaging position of the frame subject to detection relative to the imaging position of the key frame can be accurately detected on the basis of normal-line information. Therefore, an observation position can be accurately detected.

Image processing apparatus and image processing method

A polarized image acquisition sectionacquires a plurality of polarized images having different polarization directions. The polarized images show, for example, an input indicator for a user interface as a recognition target object. A normal line calculation sectioncalculates normal lines for individual pixels of the recognition target object in accordance with the polarized images acquired by the polarized image acquisition section. The normal lines represent information based on the three-dimensional shape of the recognition target object. A recognition sectionrecognizes the object by using the normal lines calculated by the normal line calculation section, determines, for example, the type, position, and posture of the input indicator, and outputs the result of determination as input information on the user interface. The object can be recognized easily and with high accuracy.

An imaging unithas a configuration in which an identical polarization pixel block made up of a plurality of pixels with an identical polarization direction is provided for each of a plurality of polarization directions and pixels of respective predetermined colors are provided in the identical polarization pixel block. A correction processing unitperforms correction processing such as white balance correction on a polarized image generated by the imaging unit. A polarized image processing unitseparates or extracts a reflection component using the polarized image after the correction processing. By using a polarized image of the separated or extracted reflection component, for example, it is possible to generate normal line information with high accuracy.

Image processing apparatus, image processing method, and image pickup element for separating or extracting reflection component

There is provided an image processing apparatus including a stereo matching unit configured to obtain right and left disparity images by using stereo matching, based on a pair of images captured by right and left cameras, respectively, a filter processing unit configured to perform filter processing on the disparity images, and a first merging unit configured to make a comparison, in the disparity images that have undergone the filter processing, between disparity values at mutually corresponding positions in the right and left disparity images and to merge the disparity values of the right and left disparity images based on a comparison result.

Growing community of inventors

Kanagawa, Japan

Ayaka Nakatani