Brainlab Ag

Christoffer Hamilton

Nils Frielinghaus

Wolfgang Steinle

Sven Flossmann

Stefan Vilsmeier

Samuel Kerschbaumer

Sebastian Stopp

Christian Schmaler

Juliane Weinzierl

Susanne Hager

Dominik Fischer

Disclosed are computer-implemented methods of training a learning algorithm on the basis of audio and image data which has been pre-processed to generate a time-synchronized transcript of the audio information and the image information to allow the learning algorithm to identify a medical device such as a medical instrument or a medical instrument on an instrument table which is visible in the image data and output corresponding information to a user. Embodiments include additional prediction of a next instrument to be used and a counting of instruments which have been used.

Training and using a learning algorithm using a transcript of audio data for identification of a medical device visible in image data

The present invention relates to a method for registering a virtual representation of a patient anatomy with a coordinate system of a physical patient anatomy, comprising displaying first overlap data in a head-mounted device, wherein the first overlap data describe from a first perspective onto the physical patient anatomy a first visual overlap between the virtual representation of the patient anatomy and the physical patient anatomy, identifying at least a first area in the first visual overlap and/or at least a first anatomical feature of the patient anatomy in the first visual overlap having at least a minimum degree of alignment between the virtual representation and the physical patient anatomy, displaying second overlap data in the head-mounted device, wherein the second overlap data describe from a second perspective onto the physical patient anatomy a second visual overlap between the virtual representation of the patient anatomy and the physical patient anatomy, and taking into account, during the displaying of the second overlap data, the alignment of the identified first area and/or anatomical feature of the patient anatomy that was identified in the first visual overlap as having at least a minimum degree of alignment between the virtual representation and the physical patient anatomy.

Method for registering a virtual representation of a patient anatomy with a coordinate system of a physical patient anatomy

A computer implemented method for determining a configuration of a medical robotic arm, wherein the configuration comprises a pose of the robotic arm and a position of a base of the robotic arm, comprising the steps of: acquiring treatment information data representing information about the treatment to be performed by use of the robotic arm; acquiring patient position data representing the position of a patient to be treated; and calculating the configuration from the treatment information data and the patient position data.

Determining a configuration of a medical robotic arm

The disclosed method encompasses registering an augmented reality device such as augmented reality glasses with a tracking coordinate system associated with a position tracking system. This may be effected by different approaches, for example by using a distance measurement unit (depth sensor) of the augmented reality device to determine a position of the augmented reality device relative to an object in a medical environment such as in surgery or radiotherapy/radiosurgery. The object may additionally be tracked by the position tracking system so that on the basis of the distance measurement, a relative position between the augmented reality device and the tracking coordinate system can be determined in order to register the augmented reality device with the position tracking system. This registration allows displaying augmentation information in a desired context and/or at a desired location in the image of the medical environment captured by the augmented reality device.

Medical augmented reality navigation

The disclosed method encompasses using an augmented reality device to blend in augmentation information including for example atlas information. The atlas information may be display separately from or in addition to a patient image (planning image). In order to display the atlas information in a proper position relative to the patient image, the two data sets are registered to one another. This registration can serve for generating a diversity of atlas-based image supplements, for example alternatively or additionally to the foregoing for displaying a segmentation of the patient image in the augmented reality image. The disclosed method is usable in a medical environment such as for surgery or radiotherapy.

Augmented reality patient positioning using an atlas

The present invention relates to a computer-implemented medical method, a computer program and a medical system () for providing, in a medical environment, an AR-overlay that is projected into the field of view provided by an AR-device () as a function of the characteristics of a user's () view with respect to an observed object ().

Visualization of medical data depending on viewing-characteristics

Provided is a method that encompasses the determination of a virtual trajectory within a virtual representation of a patient's anatomy, wherein the trajectory is defined by a user's visual axis relative to the three-dimensional virtual representation of the patient's anatomy. The method includes acquiring image data which describes the three-dimensional virtual representation of the patient's body part, acquiring position data which describes a spatial position of a user's visual axis within a virtual-world co-ordinate system, determining visualization data based on the image data and the position data, and determining virtual-world trajectory data based on the position data.

Virtual trajectory planning

The disclosed method encompasses pre-registering an anatomical body part with a coordinate system used by an augmented reality device (such as augmented reality glasses) for outputting (e.g. displaying or projecting) augmentation information. An example of the augmentation information is the position (in the real image captured by the augmented reality device) of a fine registration area on the anatomical body part which a user is supposed to identify for fine registration of the anatomical body part with a tracking coordinate system used by a medical position tracking system. The disclosed method is usable in a medical environment such as for surgery or radiotherapy.

Augmented reality pre-registration

The invention relates to a computer-implemented medical method for determining a virtual flight-path () with respect to a virtual representation () of at least one anatomical structure, the method comprising executing, on a processor of a computer, the steps of:&#x2014;acquiring, on the processor, patient image data describing at least one patient image showing at least one anatomical structure of a patient; &#x2014;acquiring, on the processor, atlas data describing at least one model of the at least one anatomical structure; &#x2014;determining, by the processor and based on the patient image data and the atlas data, representation data describing a virtual representation of the at least one anatomical structure; &#x2014;acquiring, on the processor, requirement data describing at least one requirement for at least one flight-path (); and&#x2014;determining, by the processor and based on the representation data and the requirement data, flight-path data describing at least one virtual flight-path () with respect to the virtual representation () of the at least one anatomical structure.

Atlas-based calculation of a flight-path through a virtual representation of anatomical structures

The disclosed method encompasses using an augmented reality device to blend in augmentation information including for example atlas information. The atlas information may be display separately from or in addition to a patient image (planning image). In order to display the atlas information in a proper position relative to the patient image, data the two data sets are registered to one another. This registration can serve for generating a diversity of atlas-based image supplements, for example alternatively or additionally to the foregoing for displaying a segmentation of the patient image in the augmented reality image. The disclosed method is usable in a medical environment such as for surgery or radiotherapy.

The present invention relates to a computer-implemented medical method, a computer program and a system () for controlling a display () in a medical environment based a user's () viewing direction and gestures performed by the user (), particularly involving the use of AR-goggles ().

Gesture control of medical displays

A computer implemented method for determining a configuration of a medical robotic arm, wherein the configuration comprises a pose of the robotic arm and a position of a base of the robotic arm, comprising the steps of: &#x2014;acquiring treatment information data representing information about the treatment to be performed by use of the robotic arm; &#x2014;acquiring patient position data representing the position of a patient to be treated; and &#x2014;calculating the configuration from the treatment information data and the patient position data.

Provided is a reference array holder, including a first segment configured to be mounted to a reference structure, and a second segment supporting a reference array having at least one tracking marker, wherein the second segment is movably coupled to the first segment via at least one coupling, and wherein the at least one coupling includes a sensor that provides information as to the spatial position of the second segment relative to the first segment. Also provided is tracking and/or navigation system including such reference array holder and to a computer implemented method of controlling a reference array holder.

Growing community of inventors

Aschheim, Germany

Christoffer Hamilton