Toyota Jidosha Kabushiki Kaisha

Toyota Motor Europe

The University of Edinburgh

Technical University of Munich

Inivation Ag

Sven Meier

Fabian Manhardt

Norimasa Kobori

Federico Tombari

Oisin MacAodha

Nassir Navab

Gemma Taverni

Daniel Olmeda Reino

Wadim Kehl

Luca Minciullo

Kei Yoshikawa

Octave Mariotti

Hakan Bilen

Christian Rupprecht

Diego Martin Arroyo

Manuel Nickel

Dimitri Accad

Lennart Wagler

Pengyuan Wang

Benjamin Busam

Aziz Baibabaev

Jakob Knapp

Lorenzo Garattoni

A computer-implemented method of estimating a 6D pose and shape of one or more objects from a 2D image, comprises the steps of: detecting, within the 2D image, one or more 2D regions of interest, each 2D region of interest containing a corresponding object among the one of more objects; cropping out a corresponding pixel value array, coordinate tensor, and feature map for each 2D region of interest; concatenating the corresponding pixel value array, coordinate tensor, and feature map for each 2D region of interest; and inferring, for each 2D region of interest, a 4D quaternion describing a rotation of the corresponding object in the 3D rotation group, a 2D centroid, which is a projection of a 3D translation of the corresponding object onto a plane of the 2D image given a camera matrix associated to the 2D image, a distance from a viewpoint of the 2D image to the corresponding object, a size, and a class-specific latent shape vector of the corresponding object.

6D pose and shape estimation method

A method for training a first neural network to detect the viewpoint of an object visible on an image and belonging to a given category of object when this image is inputted to the first neural network, including: providing a dataset of pairs of images under different viewpoints, providing a second neural network configured to be able to deliver appearance information of an object, providing a third neural network configured to be able to deliver a synthetic image of an object of the category using appearance information and a viewpoint, jointly training the first neural network, the second neural network, and the third neural network.

Method for training neural network, system for training neural network, and neural network

A system and a method for processing an image include inputting the image to a neural network configured to: obtain a plurality of feature maps, each feature map having a respective resolution and a respective depth, perform a classification on each feature map to deliver, for each feature map: the type of at least one object visible on the image, the position and shape in the image of at least one two-dimensional bounding box surrounding the at least one object, a plurality of rotation hypotheses for the at least one object.

Method and system for processing an image by determining rotation hypotheses

A system and a method for processing an image include inputting the image to a neural network configured to: obtain a plurality of feature maps, each feature map having a respective resolution and a respective depth, perform a classification on each feature map to deliver, for each feature map: the type of at least one object visible on the image, the position and shape in the image of at least one two-dimensional bounding box surrounding the at least one object, at least one possible viewpoint for the at least one object, at least one possible in-plane rotation for the at least one object.

Method and system for processing an image and determining viewpoints of objects

Growing community of inventors

Brussels, Belgium

Sven Meier