Metaio Gmbh

Apple Inc.

Metalo Gmbh

Selim Benhimane

Daniel R Kurz

Daniel Ulbricht

Sebastian Lieberknecht

Andrea Huber

A method of providing a descriptor for at least one feature of an image comprises the steps of providing an image captured by a capturing device and extracting at least one feature from the image, and assigning a descriptor to the at least one feature, the descriptor depending on at least one parameter which is indicative of an orientation, wherein the at least one parameter is determined from the orientation of the capturing device measured by a tracking system. The invention also relates to a method of matching features of two or more images.

Method of providing a descriptor for at least one feature of an image and method of matching features

A method for determining correspondences between a first and a second image, comprising the steps providing a first image and a second image of the real environment, defining a warping function between the first and second image, determining the parameters of the warping function between the first image and the second image by means of an image registration method, determining a third image by applying the warping function with the determined parameters to the first image, determining a matching result by matching the third image and the second image, and determining correspondences between the first and the second image using the matching result and the warping function with the determined parameters. The method may be used in a keyframe based method for determining the pose of a camera based on the determined correspondences.

Method for determining correspondences between a first and a second image, and method for determining the pose of a camera

The specification relates to determining either or both of the pose of a camera with respect to a real object or a three dimensional structure of the real object. In some of the described embodiments, a captured image including the real object is analyzed with respect to a reference image that also includes the real object. A parameter set is estimated by using a homography between the reference image and the captured image. An update to the parameter set is made by using an iterative minimization process that includes comparing a first set of pixels in the reference image with a computed set of pixels in the captured image, where the computed set of pixels used for comparison varies in different iterations of the iterative minimization process.

Method for determining a parameter set designed for determining the pose of a camera and/or for determining a three-dimensional structure of the at least one real object

A method for estimating a camera motion and for determining a three-dimensional model of an environment is provided that includes the steps of: providing intrinsic parameters of a camera; providing a set of reference two-dimensional imaged points captured by the camera at a first camera pose and reference depth samples; determining a three-dimensional model of the environment; providing a set of current two-dimensional imaged points captured by the camera at a second camera pose and current depth samples associated to the set of current two-dimensional imaged points and determining a current three-dimensional model; estimating a camera motion between the first camera pose and the second camera pose; determining a similarity measure between the three-dimensional model and the current three-dimensional model, and if it is determined that the similarity measure meets a first condition, updating the three-dimensional model of the environment and adding the set of current two-dimensional imaged points to the set of reference two-dimensional imaged points.

Method for estimating a camera motion and for determining a three-dimensional model of a real environment

The invention is related to a method for registering at least one part of a first image and of a second image, including the steps of providing a first image of the real environment and a coordinate system associated to the first image, a second image of the real environment, and measurements of orientation and/or distance of a capturing device with respect to the real environment while capturing the second image. A collineation warping function is then determined from the measurements associated to at least one part of the second image. The method further includes the steps of transforming the at least one part of the second image by applying the corresponding determined collineation warping function, determining a second warping function for registering the transformed at least one part of the second image and the at least one part of the first image by means of an image registration method, and registering the at least one part of the second image and first image using the collineation warping function and the second warping function.

Method for registering at least one part of a first and second image using a collineation warping function

A method for determining a parameter set which is designed to be used for determining the pose of a camera with regard to at least one real object and/or for determining a three-dimensional structure of the at least one real object, comprises the steps of providing a reference image including at least a part of the at least one real object, capturing at least one current image including at least a part of the at least one real object, providing an initial estimate of a parameter set which is including at least the three-dimensional translation in the common coordinate system between the pose of the camera when capturing the reference image and the pose of the camera when capturing the current image, and the depth of at least a first point of the at least one real object in the common coordinate system, and determining an update of the estimate of the parameter set by means of an iterative minimization process, wherein in the iterative minimization process a first set of pixels in the reference image is compared with a computed set of pixels in the current image and the computed set of pixels in the current image used for the comparison varies at each iteration.

A method of determining reference features for use in an optical object initialization tracking process is disclosed, said method comprising the following steps: a) capturing at least one current image of a real environment or synthetically generated by rendering a virtual model of a real object to be tracked with at least one camera and extracting current features from the at least one current image, b) providing reference features adapted for use in an optical object initialization tracking process, c) matching a plurality of the current features with a plurality of the reference features, d) estimating at least one parameter associated with the current image based on a number of current and reference features which were matched, and determining for each of the reference features which were matched with one of the current features whether they were correctly or incorrectly matched, e) wherein the steps a) to d) are processed iteratively multiple times.

Method of determining reference features for use in an optical object initialization tracking process and object initialization tracking method

A method of determining reference features for use in an optical object initialization tracking process is disclosed, said method comprising the following steps: a) capturing at least one current image of a real environment or synthetically generated by rendering a virtual model of a real object to be tracked with at least one camera and extracting current features from the at least one current image, b) providing reference features adapted for use in an optical object initialization tracking process, c) matching a plurality of the current features with a plurality of the reference features, d) estimating at least one parameter associated with the current image based on a number of current and reference features which were matched, and determining for each of the reference features which were matched with one of the current features whether they were correctly or incorrectly matched, e) wherein the steps a) to d) are processed iteratively multiple times, wherein in step a) of every respective iterative loop a respective new current image is captured by at least one camera and steps a) to d) are processed with respect to the respective new current image, and f) determining at least one indicator associated to reference features which were correctly matched and/or to reference features which were incorrectly matched, wherein the at least one indicator is determined depending on how often the respective reference feature has been correctly matched or incorrectly matched, respectively.

Growing community of inventors

Munich, Germany

Selim Benhimane