Canon Kabushiki Kaisha

Canon Europa N.v.

Alexander Ralph Lyons

Adam Michael Baumberg

Richard Ian Taylor

Simon Michael Rowe

Qi He Hong

Allan Joseph Davison

Jane Haslam

Aaron William Christopher Kotcheff

Kenichi Nagasawa

Takashi Saito

Richard A Kirk

Charles Stephen Wiles

Alexandre Percival Dodge

Koichi Matsumura

Jonathan Mark Sewell

This invention introduces, as one aspect, an apparatus for creating three-dimensional object model, comprising photographing means for photographing an object to be modeled for obtaining images to be used for creating the three-dimensional object model, setting means for longitudinally and latitudinally setting a relative position between the object and said photographing means, said setting means being capable of setting the object and said photographing means a plurality of different relative longitudinal and latitudinal positions, and control means for controlling said photographing means and said setting means so that a number of photographs taken from different relative longitudinal positions at a first relative latitudinal position is larger than that taken from different relative longitudinal positions at a second relative latitudinal position, the first relative latitudinal position being closer to a lateral position than the second relative latitudinal position. Accordingly, it becomes possible to minimize the number of photographs for obtaining necessary images to effectively keep a high quality three-dimensional object model, the number of user's manual operations is reduced, and the total photographing time for creating a three-dimensional object model is significantly decreased.

Photographing apparatus, device and method for obtaining images to be used for creating a three-dimensional model

A plurality of texture maps for a 3D computer model of an object are generated by processing images of the object, data defining the computer model, and data defining the relative positions and orientations of the images and computer model. A respective texture map is generated for the viewing position and direction of each image. Each texture map stores texture data for all parts of the computer model which are visible from the associated viewing position and direction and also texture data for all invisible parts of the computer model. Each texture map therefore stores texture data that can be used for rendering accurate images of the 3D computer model over a wide range of viewing positions and directions.

Generation of texture maps for use in 3D computer graphics

A method of generating model data representative of a model in a three-dimensional space of an object from input signals representative of a set of images of the object taken from a plurality of respective camera positions includes the steps of displaying a model image derived from the model data and having a plurality of primitives for viewing by a user, receiving at least one primitive selection signal responsive to user actuation of an input whereby each primitive selection signal identifies a respective selected primitive of the model, and defining a plurality of virtual cameras in the three dimensional space having positions and look directions relative to the model which correspond substantially to those of the respective actual cameras relative to the object. Additional steps include evaluating which of the virtual cameras is an optimum virtual camera for generating a view of the selected primitives, identifying from the camera images a first camera image of the plurality of camera images taken from a camera position corresponding to that of the optimum virtual camera, and determining from the camera images a second camera image as being suitable for matching features in the first camera image and displaying the second camera image for comparison by the user with the first camera image.

Image processing apparatus

Customer computer processing apparatus () are connected to a service apparatus () via the Internet (). Service apparatus () processes requests and payment details from a customer apparatus (), and, in response to a valid payment, sends instructions to the customer apparatus to enable the customer apparatus to control a printer () or a display panel () to print or display a photographic mat (). Service apparatus () retains details of the photographic mat pattern for subsequent use. Images of a subject object on the photographic mat are recorded at different positions and orientations with a camera (). Data defining the images is returned from the customer processing apparatus () to the service apparatus (). The service apparatus processes the image data along with the stored pattern data to identify the photographic mat features in the images and to calculate the position and orientation at which each image was recorded. A three-dimensional computer model of the subject object is then generated using the calculated positions and orientations.

View-dependent texture images and view-independent texture data for a 3D computer model are processed to compress each view-dependent texture image by subtracting the view-dependent data and view-independent data in a common 2D image space to generate a difference image, and compression encoding the difference image. The difference images, 3D computer model and view-independent texture data are transmitted to a second apparatus, where the difference data and view-independent texture data are recombined in a common 2D image space as part of the processing to render an image.

Texture data compression and rendering in 3D computer graphics

A texture map for texturing the polygon mesh of a 3D computer model during rendering is generated by defining a respective triangle within the texture map for each triangle in the polygon mesh to create a texture coordinate map, and allocating image data to each defined triangle. To generate the texture coordinate map, the triangles are defined so that the area of each triangle is dependent upon the content of texture data to be stored therein. More particularly, triangles required to store texture data with a relatively large amount of detail have a relatively large area and triangles which are required to store texture data with relatively little detail have a relatively small area. In this way, more area is allocated for the storage of detailed texture data, thereby reducing the amount of information which is lost from the texture data during the creation of a texture map.

Generating texture maps for use in 3D computer graphics

A texture map () for texturing a polygon mesh 3D computer model () during rendering is generated by defining a respective triangle within the texture map for each triangle in the polygon mesh to create a texture coordinate map (), and allocating image data to each defined triangle. To generate the texture coordinate map, the triangles in the polygon mesh are processed to generate a plurality of lists, each list identifying adjacent triangles in the mesh having shapes which can be packed into strips (-) of the texture coordinate map without significant shape distortion. The triangle lists are then packed into the texture coordinate map strips. No padding is defined between triangles in the same list, only between triangles of different lists. Image data is allocated to each triangle.

Apparatus and method for generating texture maps for use in 3D computer graphics

In an image processing system, a subject objectis placed on top of an object supporton a calibration objecthaving a known pattern of features thereon. Images recorded at different positions and orientations are processed to generate a three-dimensional computer model of the subject object alone or the subject object together with the calibration object. By imaging the subject objecton an object support instead of placing it directly on the calibration object, the user is provided with flexibility in the selection of the pattern and color of the calibration and the selection of the imaging positions and orientations. By providing an object supporthaving a top surface on which the subject objectsits which does not protrude from beneath the base of the subject object, the user is provided with flexibility in the selection of lighting conditions. By imaging the subject objectwith a background screenbehind it, the user is provided with flexibility in the selection of the surface on which the calibration objectis placed for imaging.

There is disclosed an apparatus and method for combining three or more texture mapstogether to generate a single, combined texture map. The texture maps are repeatedly combined together in pairs, with each pair generating a larger texture map for subsequent combination with another texture map, until only one texture map remains containing all of the original texture maps.

Creation of a computer representation of a three-dimensional object surface is described. The viewing cones for camera positions at which images of the object were taken are determined and the intersection of these viewing cones is used to define an initial three-dimensional space within which the object surface lies. This initial space is divided into voxels. Each non-occluded voxel is checked for photoconsistency by comparing the colors (or average colors) of the pixel patches in the images to which that voxel projects. Photo-inconsistent voxels are removed. A voxel may be determined to be photo-inconsistent if there exists no set of photoconsistent pixel regions or the pixel patches do not share a color value range. The pixel patch into which each voxel projects in a further image may be compared with the stored color for that voxel and any photo-inconsistent voxels removed. This process can then be repeated for further images.

To generate texture data for a 3D computer modelof a real-life objectimagesof the subject object are recorded from different viewing positions and directions. The image data is processed by an image processing apparatusto register the viewing positions and directions of the imagesbased on the positions of features in the images. The 3D computer modelof the subject object is registered with the resulting registered set of images, and texture data is generated for the 3D computer model from the images.

In an image processing apparatusan input sequenceof video images is processed to determine the different positions and orientations at which the images were recorded in an efficient and accurate manner. A subset of the input images are selected as keyframes to form a sequenceof keyframes. Respective triples of keyframes having different, non-overlapping positions in the sequenceare selected and processed to determine the relative positions and orientations at which the keyframes in each triple were recorded to form respective sets of keyframes. The positions and orientations of keyframes between the keyframes in each triple are then calculated to form expanded sets of keyframes. The sets are further expanded by calculating the positions and orientations of keyframes which lie between sets in the sequence. The sets are merged by calculating the relationship between the coordinate systems in which the positions and orientations of the keyframes in each set are defined. During the processing, the positions and orientations calculated for keyframes in a set are adjusted to optimise the calculated solutions. This is performed in stages, considering at each stage a different windowof the keyframes and performing processing to minimise the error associated with the keyframes in the window. The window is moved sequentially through the keyframes so that every keyframe in a set is considered at least once.

To generate a 3D computer model of a subject objectimagesof the subject object are recorded from different viewing positions and directions. The image data is processed to generate a plurality of sets of images, each set containing images having registered imaging positions and directions. A preliminary 3D computer modelis generated using the images from a first of the sets such that the position and orientation of the preliminary 3D computer modelis registered with the images in the set. The imaging positions and directions of the images in the first set are then registered with the imaging positions and directions of the images in each other respective set. This is done by projecting the preliminary 3D computer modelinto images from each other set, and changing the relative position and orientation of the preliminary 3D computer modeland images to align the silhouette of the projected 3D computer model with the silhouette of the imaged subject object in the images. A refined 3D computer model of the subject object is generated using images from the different sets which are registered.

Three-dimensional computer modelling

In a computer processing apparatus, a number of depth mapsof a subject objectare processed to generate a 3D computer model of the subject object. The points in each depth map are connected to give a 2D mesh, and each 2D mesh is then projected into 3D space in dependence upon the depths of the points in the mesh, thereby giving a 3D mesh. Side faces for each edge of the 3D mesh are added extending away from the depth map, thereby generating a respective polyhedronfor each depth map. The 3D computer model of the subject object is generated by calculating the intersections of the polyhedra.

3D computer modelling apparatus

A 3D computer model of an object is generated by calculating the intersections of polyhedra. Each polyhedron defines a volume of 3D space containing at least part of the object. The 3D points of intersection of the planar faces of the polyhedra are calculated and each point is labeled with the planar faces which meet thereat. The points are connected to form a polygon mesh using the labels to determine which points should be connected together. In calculating the points, a volume containing the object is subdivided into parts, each part is tested against the polyhedra and then discarded, subdivided further, or the point of intersection of planar faces within the volume part is calculated. A volume part is discarded if it is outside at least one polyhedron. The volume part is subdivided into further parts if it is intersected by more than a predetermined number of polyhedra faces.

Growing community of inventors

Bracknell, United Kingdom

Alexander Ralph Lyons