Toyota Jidosha Kabushiki Kaisha

Toyota Motor Europe

Technical University of Munich

Norimasa Kobori

Sven Meier

Fabian Manhardt

Nassir Navab

Federico Tombari

Wadim Kehl

Christian Rupprecht

Diego Martin Arroyo

Luca Minciullo

Manuel Nickel

Kei Yoshikawa

A computer-implemented method of estimating a 6D pose and shape of one or more objects from a 2D image, comprises the steps of: detecting, within the 2D image, one or more 2D regions of interest, each 2D region of interest containing a corresponding object among the one of more objects; cropping out a corresponding pixel value array, coordinate tensor, and feature map for each 2D region of interest; concatenating the corresponding pixel value array, coordinate tensor, and feature map for each 2D region of interest; and inferring, for each 2D region of interest, a 4D quaternion describing a rotation of the corresponding object in the 3D rotation group, a 2D centroid, which is a projection of a 3D translation of the corresponding object onto a plane of the 2D image given a camera matrix associated to the 2D image, a distance from a viewpoint of the 2D image to the corresponding object, a size, and a class-specific latent shape vector of the corresponding object.

6D pose and shape estimation method

A system and a method for processing an image include inputting the image to a neural network configured to: obtain a plurality of feature maps, each feature map having a respective resolution and a respective depth, perform a classification on each feature map to deliver, for each feature map: the type of at least one object visible on the image, the position and shape in the image of at least one two-dimensional bounding box surrounding the at least one object, a plurality of rotation hypotheses for the at least one object.

Method and system for processing an image by determining rotation hypotheses

Growing community of inventors

Brussels, Belgium

Norimasa Kobori