Canon Kabushiki Kaisha

Richard Ian Taylor

Alexander Ralph Lyons

Simon Michael Rowe

Jane Haslam

Allan Joseph Davison

Adam Michael Baumberg

Charles Stephen Wiles

Richard A Kirk

Aaron William Christopher Kotcheff

Alexandre Percival Dodge

In an image processing apparatus, images of a subject objectand data defining the positions and orientations at which the images were recorded are processed to generate a three-dimensional computer model of the subject object. As part of the processing, image data relating to the subject objectis segmented from other image data in each input image. The results of the segmentation processing are displayed to a user and the apparatus is operable by the user to edit the segmentation processing to correct any errors. Such errors typically occur due to the misclassification of pixels relating to shadows in an input image as part of the subject objectand the misclassification of pixels relating to features on the surface on which the subject objectis placed for imaging as part of the subject object. The facility to edit the image data segmentations therefore provides the user with greater flexibility in the lighting conditions which can be used for imaging and greater flexibility in the selection of the surface on which the subject objectis placed. The results of the segmentation processing, including any edited results, are used to generate a three-dimensional computer model of the subject object

Image processing apparatus and method for generating a three-dimensional model of an object from a collection of images of the object recorded at different viewpoints and segmented using semi-automatic segmentation techniques

A method of generating model data representative of a model in a three-dimensional space of an object from input signals representative of a set of images of the object taken from a plurality of respective camera positions includes the steps of displaying a model image derived from the model data and having a plurality of primitives for viewing by a user, receiving at least one primitive selection signal responsive to user actuation of an input whereby each primitive selection signal identifies a respective selected primitive of the model, and defining a plurality of virtual cameras in the three dimensional space having positions and look directions relative to the model which correspond substantially to those of the respective actual cameras relative to the object. Additional steps include evaluating which of the virtual cameras is an optimum virtual camera for generating a view of the selected primitives, identifying from the camera images a first camera image of the plurality of camera images taken from a camera position corresponding to that of the optimum virtual camera, and determining from the camera images a second camera image as being suitable for matching features in the first camera image and displaying the second camera image for comparison by the user with the first camera image.

Image processing apparatus

Customer computer processing apparatus () are connected to a service apparatus () via the Internet (). Service apparatus () processes requests and payment details from a customer apparatus (), and, in response to a valid payment, sends instructions to the customer apparatus to enable the customer apparatus to control a printer () or a display panel () to print or display a photographic mat (). Service apparatus () retains details of the photographic mat pattern for subsequent use. Images of a subject object on the photographic mat are recorded at different positions and orientations with a camera (). Data defining the images is returned from the customer processing apparatus () to the service apparatus (). The service apparatus processes the image data along with the stored pattern data to identify the photographic mat features in the images and to calculate the position and orientation at which each image was recorded. A three-dimensional computer model of the subject object is then generated using the calculated positions and orientations.

In an image processing apparatus, a face recognizer is trained using images of one or more faces to generate representation data for the face recognizer characterising the face(s). Face recognition processing is then performed using the representation data. The training and face recognition processing is performed taking into account a person's age.

Age-based face recognition

A gaze tracking system () is provided which obtains images of an operator () and processes the images to associate the images with coordinates of where the operators gaze is directed towards. In order to identify images where an operator is looking in the same direction, images obtained by a camera () are first processed using the retinex algorithm. Image patches from the processed image are then compared with stored feature images and an initial classification based upon the correspondence between areas of the processed image and the stored feature images is determined. This initial classification is then further processed and each image is assigned a single classification. The areas identified by different classifications are then determined so that classifications assigned to adjacent images identify points closer together and classifications never assigned to an adjacent images identify points further apart.

Gaze tracking system

In an image processing system, a subject objectis placed on top of an object supporton a calibration objecthaving a known pattern of features thereon. Images recorded at different positions and orientations are processed to generate a three-dimensional computer model of the subject object alone or the subject object together with the calibration object. By imaging the subject objecton an object support instead of placing it directly on the calibration object, the user is provided with flexibility in the selection of the pattern and color of the calibration and the selection of the imaging positions and orientations. By providing an object supporthaving a top surface on which the subject objectsits which does not protrude from beneath the base of the subject object, the user is provided with flexibility in the selection of lighting conditions. By imaging the subject objectwith a background screenbehind it, the user is provided with flexibility in the selection of the surface on which the calibration objectis placed for imaging.

Creation of a computer representation of a three-dimensional object surface is described. The viewing cones for camera positions at which images of the object were taken are determined and the intersection of these viewing cones is used to define an initial three-dimensional space within which the object surface lies. This initial space is divided into voxels. Each non-occluded voxel is checked for photoconsistency by comparing the colors (or average colors) of the pixel patches in the images to which that voxel projects. Photo-inconsistent voxels are removed. A voxel may be determined to be photo-inconsistent if there exists no set of photoconsistent pixel regions or the pixel patches do not share a color value range. The pixel patch into which each voxel projects in a further image may be compared with the stored color for that voxel and any photo-inconsistent voxels removed. This process can then be repeated for further images.

A 3-D model of an object is created by processing images taken from a series of camera positions. An initial sequence of the images is processed to define respective image co-ordinates of matching features to generate a set of model data defining model points in a 3-D space of the model and to obtain respective camera solutions representative of positions and orientations of virtual cameras in the 3-D space defining views of the model corresponding to the images. A new image is added to the sequence and processed to obtain a camera solution for a corresponding new virtual camera for use in generating further model data.

In an image processing apparatus, image data, defining input images of a scene recorded at different times and with different viewing positions and viewing directions, is processed to generate data defining a time-lapse sequence of images for display to a user showing an evolving representation of the scene between the times at which the first and last input images were recorded. Processing is carried out to register the input images by calculating the transformations which align matching features in the images, and interpolation is carried out between the pixel values of the registered images to generate pixel values for interpolated images in the time-lapse sequence. In this way, it is not necessary for the photographer to leave a camera in position to record images for a time-lapse sequence. Instead, the photographer can return to the scene as and when required to record the images since the viewing position and viewing direction for the images do not need to be the same.

In an apparatus and method for creating a three-dimensional model of an object, images of the object taken from different, unknown positions are processed to identify the points in the images which correspond to the same point on the actual object (that is &ldquo;matching&rdquo; points), the matching points are used to determine the relative positions from which the images were taken, and the matching points and calculated positions are used to calculate points in a three-dimensional space representing points on the object. A number of different techniques are used to identify the matching points, and a number of solutions are calculated and tested for the relative positions, the solution which is consistent with the largest number of matching points being selected. In one matching technique, edges in an image are identified by first identifying corner points in the image and then identifying edges between the corner points on the basis of edge orientation values of pixels, the edges are processed in strength order to remove cross-overs, the images sub-divided into regions by connecting points at the ends of the edges on the basis of the edge strengths, and matching points within corresponding regions in two or more images are identified.

In an apparatus and method for creating a three-dimensional model of an object, images of the object taken from different, unknown positions are processed to identify the points in the images which correspond to the same point on the actual object (that is &ldquo;matching&rdquo; points), the matching points are used to determine the relative positions from which the images were taken, and the matching points and calculated positions are used to calculate points in a three-dimensional space representing points on the object. A number of different techniques are used to identify the matching points, and a number of solutions are calculated and tested for the relative positions. Edges in an image are identified by first identifying corner points in the image and then identifying edges between the corner points on the basis of edge orientation values of pixels, the edges are processed in strength order to remove cross-overs, the images sub-divided into regions by connecting points at the ends of the edges on the basis of the edge strengths, and matching points within corresponding regions in two or more images are identified.

Growing community of inventors

Bracknell, United Kingdom

Richard Ian Taylor