Toyota Jidosha Kabushiki Kaisha

Technische Universitat Munchen

Stryker Leibinger Gmbh & Co Kg

Mvtec Software Gmbh

Technical University of Munich

Erbe Vision Gmbh

Nassir Navab

Stefan Hinterstoisser

Sven Meier

Fabian Manhardt

Norimasa Kobori

Luca Minciullo

Selim Ben Himane

Matthias Grimm

Kei Yoshikawa

Javier Esteban

Alexander Winkler

Alejandro Martin Gomez

Manuel Nickel

Wojciech Konrad Karcz

Pengyuan Wang

Lorenzo Garattoni

Benjamin Busam

Described herein are systems, methods, and techniques for spatially-aware displays for computer-assisted interventions. A Fixed View Frustum technique renders computer images on the display using a perspective based on a virtual camera having a field-of-view facing the display and automatically updates the virtual position of the virtual camera in response to adjusting the pose of the display. A Dynamic Mirror View Frustum technique renders computer images on the display using a perspective based on a field-of-view of a virtual camera that has a virtual position behind the display device. The virtual position of the virtual camera is dynamically updated in response to movement of a user's viewpoint located in front of the display device. Slice visualization techniques are also described herein for use with the Fixed View Frustum and Dynamic Mirror View Frustum techniques.

Spatially-aware displays for computer-assisted interventions

A computer-implemented method of estimating a 6D pose and shape of one or more objects from a 2D image, comprises the steps of: detecting, within the 2D image, one or more 2D regions of interest, each 2D region of interest containing a corresponding object among the one of more objects; cropping out a corresponding pixel value array, coordinate tensor, and feature map for each 2D region of interest; concatenating the corresponding pixel value array, coordinate tensor, and feature map for each 2D region of interest; and inferring, for each 2D region of interest, a 4D quaternion describing a rotation of the corresponding object in the 3D rotation group, a 2D centroid, which is a projection of a 3D translation of the corresponding object onto a plane of the 2D image given a camera matrix associated to the 2D image, a distance from a viewpoint of the 2D image to the corresponding object, a size, and a class-specific latent shape vector of the corresponding object.

6D pose and shape estimation method

A method for aligning the positions and orientations of a real object and a virtual object in real space, the virtual object corresponding to a virtual replica of the real object, the method comprising visualizing at least one alignment feature superimposed on or replacing a representation of the virtual object in a field of view containing the real object, wherein the alignment feature is indicative of a position and orientation of the virtual object in real space, and wherein the at least one alignment feature complements a shape and/or surface pattern of the real object, such that the alignment feature and the real object form a composite object with complementing patterns and/or shapes in the field of view, when the real object and the virtual object are aligned.

Method of aligning virtual and real objects

Growing community of inventors

Munich, Germany

Nassir Navab