Canon Kabushiki Kaisha

Tatsuro Koizumi

Shugo Higuchi

Kazuhiro Yoshimura

Tomohiro Nishiyama

Yoshinari Higaki

Kaori Taya

The information processing apparatus has an image correction unit. The image correction unit has a correction parameter determination unit configured to determine a correction parameter for correcting an image capturing characteristic of image data corresponding to a target viewpoint among a plurality of pieces of image data acquired by capturing an object from a plurality of viewpoints based on an image capturing characteristic of image data of another viewpoint different from the target viewpoint and a pixel value correction unit configured to correct a pixel value of the image data corresponding to the target viewpoint based on the correction parameter. Moreover, the information processing apparatus further has an image composition unit configured to generate composed image data based on the image data whose pixel value has been corrected.

Information processing apparatus, information processing method and non-transitory computer readable storage medium

A system comprises an obtainment unit that obtains virtual viewpoint information relating to a position and direction of a virtual viewpoint; a designation unit that designates a focus object from a plurality of objects detected based on at least one of the plurality of images captured by the plurality of cameras; a decision unit that decides an object to make transparent from among the plurality of objects based on a position and direction of a virtual viewpoint that the virtual viewpoint information obtained by the obtainment unit indicates, and a position of the focus object designated by the designation unit; and a generation unit that generates, based on the plurality of captured images obtained by the plurality of cameras, a virtual viewpoint image in which the object decided by the decision unit is made to be transparent.

Image processing apparatus that generates a virtual view image from multiple images captured from different directions and method controlling the same

An image processing apparatus includes: an image obtaining unit configured to obtain images of an image capturing target area from a plurality of directions; an information obtaining unit configured to obtain viewpoint information; a determination unit configured to determine a pixel value of a particular area in a virtual viewpoint image according to a virtual viewpoint at a first time indicated by the viewpoint information, based on a pixel value of an obtained image based on image capturing by a first image capturing apparatus at a second time, and a pixel value of an obtained image based on image capturing by a second image capturing apparatus at the first time, wherein the particular area is not captured by the first image capturing apparatus at the first time; and a generation unit configured to generate a virtual viewpoint image in accordance with the determination of the pixel value.

Image processing apparatus, image processing method, and storage medium

Coordinates on a two-dimensional image that respectively correspond to points on a three-dimensional space that configures an object included in a field of view based on a viewpoint are specified. A depth map is generated based on a result of assignment processing that assigns distance information of a first point corresponding to first coordinates to a first area including the first coordinates and assigns distance information of a second point corresponding to second coordinates to a second area including the second coordinates, the first and second coordinates belonging to coordinates on the two-dimensional image. A distance from the viewpoint to the first point is longer than a distance from the viewpoint to the second point, and the first area is smaller than the second area. The points configuring the object include the first and second points.

Image processing apparatus, generation method, and non-transitory computer-readable storage medium

There is provided with an image processing apparatus. A determination unit determines color information of an object present in an object image from a virtual viewpoint. The color information of the object is determined by combining color information of the object in a captured image obtained by one or more image capturing apparatus selected based on position and orientation information of the virtual viewpoint, position and orientation information of the image capturing apparatuses, and position information of the object, by using a weight corresponding to an image capturing direction of the one or more image capturing apparatus and a position of the object in a field of view of the one or more image capturing apparatus.

Image processing apparatus, image processing method, and medium

The invention comprises a depth-image generator which, based on shape information representing a shape of an object in the captured images of the cameras, generates depth images corresponding to the captured images obtained from a plurality of cameras; a detector which, based on a generated depth image, detects a boundary region of the object in a corresponding captured image; a pixel map generator which, based on the boundary region in the captured image detected by the detector, generates a pixel map representing a contributing ratio for generating the virtual-viewpoint image for each pixel position of the captured image; and an output-image generator which generates the virtual-viewpoint image based on the captured images obtained from the plurality of cameras, and the pixel map corresponding to the captured images.

Image processing apparatus and method of controlling the same

An image processing apparatus includes an acquisition unit, first and second correction units, first and second calculation units, and a determination unit. The acquisition unit acquires image data pieces obtained by temporally successively capturing images of an object. The first correction unit acquires first correction data by performing fluctuation correction on processing target image data using temporally neighboring image data pieces among the acquired image data pieces. The first calculation unit calculates, as first displacement data, an amount of displacement between reference and the processing target image data or the first correction data. The determination unit determines a moving object region. The second calculation unit calculates second displacement data by interpolating the first displacement data in the determined moving object region based on the first displacement data or the first correction data. The second correction unit corrects based on the second displacement data to obtain second correction data.

An image processing apparatus includes: an acquisition unit configured to acquire pieces of silhouette image data of an object viewed from multiple different viewpoints; a generation unit configured to generate, from the pieces of silhouette image data, pieces of low-resolution data representing images with a resolution lower than the pieces of silhouette image data; and an estimation unit configured to, by performing, for a plurality of line segments in space containing the object, processing in which, after a line segment in the space is projected onto a piece of low-resolution data to calculate a first intersection of the line segment with the object, the line segment is projected onto a piece of silhouette image data to calculate a second intersection of the line segment with the object, calculate intervals over which the plurality of line segments intersect the object and estimate a shape of the object.

Image processing apparatus, image processing method, and non-transitory storage medium

The image processing apparatus according to an embodiment of the present invention corrects a color blur of a refocus image due to light field data. The image processing apparatus includes a generation unit configured to generate a refocus image for each channel at a virtual refocus position based on light field data including a plurality of channels whose wavelengths are different and optical characteristics for each channel of an optical system used to obtain the light field data. Further, the image processing apparatus includes a derivation unit configured to derive spatial frequency characteristics for each channel from the optical characteristics for each channel and the virtual refocus position and a correction unit configured to correct the refocus image for each channel so as to make sharpness uniform based on the spatial frequency characteristics for each channel.

Image processing apparatus, image capturing apparatus, image processing method, and storage medium

An image processing apparatus includes an acquisition unit, a first calculation unit, a combination unit, a second calculation unit, and a correction unit. The acquisition unit acquires a plurality of pieces of image data captured sequentially in time series. The first calculation unit calculates a first displacement between one piece of image data among the plurality of pieces of image data and a piece of image data adjacent in time series to the one piece of image data. The combination unit combines a plurality of first displacements to obtain a second displacement between the one piece of image data and a piece of image data neighboring the one piece of image data in time series. The second calculation unit calculates a correction amount for the one piece of image data based on the second displacement. The one piece of image data is corrected based on the calculated correction amount.

In a motion picture capturing system for generating a virtual viewpoint image by a plurality of cameras, if image-capturing is performed with timing being shifted to improve frame rate performance, estimation accuracy is deteriorated due to an error in corresponding point detection between images or the presence of a moving subject. The image-capturing timing is controlled such that two or more of a plurality of image-capturing units perform simultaneous image-capturing, and motion of the whole of an image-capturing apparatus is estimated from a plurality of images captured by the simultaneous image-capturing. The virtual viewpoint image is synthesized from the images captured by the simultaneous image-capturing by the plurality of image-capturing units and the estimated motion.

Image-capturing apparatus

Capturing units acquire, from an object, image data having characteristics different from each other. An image generator calculates a deviation amount indicating the position deviation between images represented by the image data acquired by the capturing units, and generates image data of a multi-channel image in which the position deviation between the images represented by the image data acquired by the capturing units has been corrected using the deviation amount. Each of the capturing units comprises capturing elements having capturing characteristics specific to the capturing unit, and capturing elements having capturing characteristics common to a plurality of capturing units. The image generator calculates the deviation amount from image data acquired by the capturing elements having the common capturing characteristics.

Growing community of inventors

Niiza, Japan

Tatsuro Koizumi