Siemens Corporate Research, Inc.

Johns Hopkins University

Siemens Aktiengesellschaft

Other

Siemens Medical Solutions Usa, Inc.

Surgiceye Gmbh

Siemens Corporation

Areva Np Gmbh

Technische Universitat Munchen

Toyota Motor Europe

Mvtec Software Gmbh

Siemens Corporation Research, Inc.

Technische Universitat Munchen Forschungsforderung Und Technologietransfar

Maxer Endoscopy Gmbh

Nassir Navab

Ali Reza Bani-Hashemi

Bernhard Geiger

Yakup Genc

Matthias Mitschke

Frank Sauer

Benedicte Bascle

Peter Kazanzides

Xiang Zhang

Ehsan Azimi

Long Qian

Thomas Wendler

Mirko Appel

Vijay Kumar

Mihran Tuceryan

Javad Fotouhi

Bernhard Fuerst

Rainer Graumann

Stefan Hinterstoisser

Chenyang Xu

Ali Khamene

James P Williams

Wolfgang Friedrich

Karl Wiesent

Soeren Moritz

Ali-Reza Bani-Hashemi

Mathias Unberath

Noah Cowan

Jochen Zeltner

Wadim Kehl

Benjamin Glocker

Tobias Sielhorst

Eric S Fortune

Pierre Georgel

Greg M Osgood

Jörg Traub

Ralf Keller

Victor Antonio Castaneda Zeman

Erin McGarrity

Diana Mateus

Nicholas Craft

Alexander Winkler

Matthias Grimm

Javier Esteban

Wojciech Konrad Karcz

Selim Ben Himane

Christian Wachinger

Zbigniew Wasik

Christoph Bichlmeier

A device may receive, from an imaging device, a two-dimensional image of a patient being operated on by a user, where the two-dimensional image captures a portion of the patient, and where the portion of the patient is provided between a focal point of an imaging source of the imaging device and a detector plane of the imaging device. The device may translate the two-dimensional image along a frustum of the imaging source, and may generate one or more images in a three-dimensional space based on translating the two-dimensional image along the frustum of the imaging source. The device may provide the one or more images in the three-dimensional space to an augmented reality device associated with the user.

Interactive flying frustums visualization in augmented reality

A calibration platform may obtain measurements for aligning a real-world coordinate system and a display coordinate system. For example, the calibration platform may display, via an optical see-through head-mounted display (OST-HMD), a three-dimensional virtual object and receive, from a positional tracking device, information that relates to a current pose of a three-dimensional real-world object to be aligned with the three-dimensional virtual object. The calibration platform may record a three-dimensional position of a plurality of points on the three-dimensional real-world object based on the current pose of the three-dimensional real-world object, based on an indication that the plurality of points on the three-dimensional real-world object respectively corresponds with a plurality of points on the three-dimensional virtual object. Accordingly, based on the obtained measurements, the calibration platform may generate a transformation function to provide a mapping between three-dimensional points in the real-world coordinate system and three-dimensional points in the display coordinate system.

Calibration system and method to align a 3D virtual scene and a 3D real world for a stereoscopic head-mounted display

A calibration platform may display, via an optical see-through head-mounted display (OST-HMD), a virtual image having at least one feature. The calibration platform may determine, based on information relating to a gaze of a user wearing the OST-HMD, that the user performed a voluntary eye blink to indicate that the at least one feature of the virtual image appears to the user to be aligned with at least one point on the three-dimensional real-world object. The calibration platform may record an alignment measurement based on a position of the at least one point on the three-dimensional real-world object in a real-world coordinate system based on a time when the user performed the voluntary eye blink. Accordingly, the alignment measurement may be used to generate a function providing a mapping between three-dimensional points in the real-world coordinate system and corresponding points in a display space of the OST-HMD.

Blink-based calibration of an optical see-through head-mounted display

A computer-implemented method for registering low dimensional images with a high dimensional image includes receiving a high dimensional image of a region of interest and simulating synthetic low dimensional images of the region of interest from a number of poses of a virtual low dimensional imaging device, from the high dimensional image. The method determines positions of landmarks within the low dimensional images by applying a first learning algorithm to the low dimensional images and back projecting of the positions of the determined landmarks into the high dimensional image space, to thereby obtain the positions of the landmarks in the high dimensional image. The positions of landmarks within low dimensional images acquired form an imaging device are determined by applying the first or a second learning algorithm to the low dimensional images. The low dimensional images are registered with the high dimensional image based on the positions of the landmarks.

Computer-implemented method for registering low dimensional images with a high dimensional image, a method for training an aritificial neural network useful in finding landmarks in low dimensional images, a computer program and a system for registering low dimensional images with a high dimensional image

A system for modelling a portion of a patient includes a processing unit, a manipulator, a sensor, and a display device. The processing unit is configured to receive patient data and to process the patient data to generate a model of the portion of the patient. The sensor is configured to capture manipulator data. The processing unit is configured to receive the manipulator data from the sensor and to process the manipulator data to determine a position of the manipulator, an orientation of the manipulator, or both. The display device is configured to display the model on or in the manipulator.

Tactile augmented reality for medical interventions

A method for aligning a real-world object with a virtual object includes capturing images, video, or both of the real-world object from a first viewpoint and from a second viewpoint. The first and second viewpoints are different. The method also includes simultaneously superimposing the virtual object at least partially over the real-world object from the first viewpoint in a first augmented reality (AR) display and from the second viewpoint in a second AR display based at least in part on the images, video, or both. The method also includes adjusting a position of the real-world object to at least partially align the real-world object with the virtual object from the first viewpoint in the first AR display and from the second viewpoint in the second AR display.

Calibration techniques for aligning real-world objects to virtual objects in an augmented reality environment

A medical imaging apparatus for combined X-ray and optical visualization is provided. It comprises: an X-ray detector positioned above a patient; an X-ray source positioned below a patient; a control device; and a camera setup adapted to deliver an optical stereoscopic or 3D image. Thereby, the camera setup is positioned adjacent to the X-ray detector above the patient, and the control device is adapted to calculate an optical 2D image or a 3D surface from the data delivered by the camera setup, that optical 2D image or 3D surface having a virtual viewpoint similar to the viewpoint of the X-ray source. It is further adapted to superimpose an X-ray image acquired by the X-ray detector and the optical 2D image or 3D surface in order to achieve an augmented optical/X-ray image.

Apparatus and method for augmented visualization employing X-ray and optical data

A method and system for scanning an object using an RGB-D sensor, the method includes: a plurality of elementary scans of the object using an RGB-D sensor and visual odometry, each elementary scan delivering a plurality of key frames associated with a pose of the sensor with respect to the object, and each elementary scan being associated with a position of the object; for each elementary scan, elaborating a three-dimensional model of the object using the plurality of key frames and poses of the scan; merging each three-dimensional model into a merged three-dimensional model of the object.

Method and system for scanning an object using an RGB-D sensor

A system for tracking a tracking element inside a body is provided. It includes a first tracking system, which includes a first tracking sensor for detecting the current position of a tracking element inside a body relative to a nominal position with respect to the first tracking sensor, and a second tracking system, which includes a second tracking sensor for determining the position of the first tracking sensor of the first tracking system relative to the second tracking sensor, and a control unit. The first tracking sensor is movable, and the control unit is configured to deliver a correction signal indicating a deviation of the current position of the tracking element relative to a nominal position with reference to the first tracking sensor. Further, a respective tracking method is provided.

Tracking system and method

A system and method for tracking a position of an interventional medical device configured to be deployed into a subject during a medical procedure. An anatomical dataset of the subject is acquired while at least one electrical sensor is disposed along the interventional medical device. A feedback signal is generated from the electrical sensor and is processed and analyzed to identify positional information of the interventional medical device. A position of the interventional medical device with respect to the anatomical dataset is identified and the position of the interventional medical device with respect to the anatomical dataset is shown on a display.

System and method for bioelectric localization and navigation of interventional medical devices

An image generating apparatus for image generation is provided. The image generating apparatus includes a movable detector for detecting nuclear radiation during a detection period and an evaluation system. The evaluation system includes an interface system for transmitting detector data to the evaluation system. The detector data include information about the detected radiation for image generation. The evaluation system further includes a data memory portion for storing the detector data. The evaluation system further includes a program memory portion with a program for repeatedly determining at least one quality value with respect to image generation during the detection period.

Image formation apparatus and method for nuclear imaging

The present invention provides a method and apparatus for depth image acquisition of an object wherein at least two time-of-flight cameras are provided and the object is illuminated with a plurality of different light emission states by emitting light signals from each camera. Measurements of the light reflected by the object during each light emission state are obtained at all the cameras wherein the measurements may then be used to optimize the depth images at each of the cameras.

Method of enhanced depth image acquisition

A device for combined optical and nuclear image comprises a nuclear image acquisition module and a reference image acquisition module. The reference image acquisition module has: an optical image sensor; and an optical imaging system for deflecting the optical radiation from a reference field of view to the image sensor, wherein the optical imaging system comprises a mirror for the optical radiation, which mirror is arranged between the reference field of view and the optical image sensor. On the image sensor, a respective image area assigned to at least one of nuclear partial fields of view. The optical imaging system is arranged such that the optical radiation coming from the at least one nuclear partial field of view is deflected substantially exactly to the respectively assigned of the image areas.

Device and method for combined optical and nuclear image acquisition

The CV-CAD (computer vision-computer-aided design) model is an enhanced CAD (computer-aided design) model that integrates local and global computer vision data in order to represent an object not only geometrically but also in terms of computer vision. The CV-CAD model provides a scalable solution for intelligent and automatic object recognition, tracking and augmentation based on generic models of objects.

Computer vision CAD model

Growing community of inventors

Plainsboro, NJ, United States of America

Nassir Navab