Sentiar, Inc.

Washington University

Istituti Clinici Scientifici Maugeri Spa Sb

Excera Inc.

Jonathan R Silva

Jennifer N Avari Silva

Michael K Southworth

Walter M Blume

Ignacio Soriano

Christopher Michael Andrews

Kristen Naegle

Zahid Iqbal

Alexander Bair Henry

Wandi Zhu

Silvia Priori

Andrea Mazzanti

Christopher Andrews

Sharif Razzaque

Matthew Dagley

Irene Hougard

Berk Tas

Walter Blume

In an embodiment, a system receives data from a first electromagnetic sensor coupled to a head-mounted display (HMD) and detecting an electromagnetic field generated by an electromagnetic reference source coupled to an ultrasound probe. The system receives data from a second electromagnetic sensor coupled to a medical instrument and detecting the electromagnetic field. The system determines a position of the HMD relative to the ultrasound probe. The system determines a position of the medical instrument relative to the ultrasound probe. The system generates a visualization of a path of the medical instrument oriented relative to an ultrasound image plane. The system provides a graphic for display by the HMD to a user wearing the HMD, where the graphic includes the visualization and image data captured by the ultrasound probe displayed on the ultrasound image plane.

Multiscale ultrasound tracking and display

A mixed reality (MR) visualization system includes an MR device comprising a holographic display configured to display a holographic image to an operator, a hand-held ultrasound imaging device configured to obtain a real-time ultrasound image of a subject's anatomy, and a computing device communicatively coupled to the MR device and the hand-held ultrasound imaging device. The computing device includes a non-volatile memory and a processor. The computing device is configured to receive the real-time ultrasound image, determine a real-time 3D position and orientation of the hand-held ultrasound imaging device, generate a modified real-time ultrasound image by modifying the real-time ultrasound image to correspond to the real-time 3D position and orientation of the hand-held ultrasound imaging device, and transmit the modified real-time ultrasound image to the MR device for display as the holographic image positioned at a predetermined location relative to the hand-held ultrasound imaging device.

System and method for augmented reality data interaction for ultrasound imaging

A processing system provides two-way communication between an electroanatomic navigation and mapping system (EAMS) and a head-mounted display (HMD). A user wearing the HMD can use hands-free or touch-free actions to interact with a user interface for EAMS information displayed by the HMD (e.g., 3D augmented reality graphics) or by a display monitor (e.g., 2D graphics). For example, the processing system updates the position of a cursor on the display monitor to track a gaze direction of the user as determined by the HMD.

Two-way communication between head-mounted display and electroanatomic system

Disclosed herein is a method for predicting a patient response to a sodium ion channel blocker such as mexiletine when the patient has LQT syndrome or an arrhythmia. The method generally comprises determining a plurality of parameters associated with sodium ion channels; generating a model for patient response by using a partial least squared (PLS) regression analysis on said plurality of parameters; and using the model to predict the patient response if the patient is administered a sodium ion channel blocker such as mexiletine.

Predicting patient response to sodium channel blockers

In an embodiment, a processing system provides an augmented reality object for display by a head-mounted device (HMD) worn by a user. The processing system provides an augmented reality graphic for display by the HMD on a plane and overlaid on the augmented reality object. The processing system determines a gaze direction of the user using sensor data captured by a sensor of the HMD. Responsive to determining that the gaze direction intersects with the augmented reality graphic on the plane and remains intersecting for at least a period of time, the processing system determines a position of intersection between the gaze direction and the augmented reality graphic on the plane. The processing system provides a modified version of the augmented reality object for display by the HMD according to the position of intersection during the period of time.

Gaze based interface for augmented reality environment

Systems and methods for virtual reality or augmented reality (VR/AR) visualization of 3D medical images using a VR/AR visualization system are disclosed that includes a computing device operatively coupled to a VR/AR device, which includes a holographic display and at least one sensor. The holographic display is configured to display a holographic image to an operator. The computing device is configured to receive at least one stored 3D image of a subject's anatomy and at least one real-time 3D position of at least one surgical instrument, to register the at least one real-time 3D position of the at least one surgical instrument to correspond to the at least one 3D image of the subject's anatomy, and to generate the holographic image comprising the at least one real-time position of the at least one surgical instrument overlaid on the at least one 3D image of the subject's anatomy.

System and method for virtual reality data integration and visualization for 3D imaging and instrument position data

In an embodiment, an electrogram (EGM) processing system provides, for display by a head-mounted display (HMD) worn by a user, a holographic rendering of intracardiac geometry. The HMD also displays an electrogram waveform. The EGM processing system determines a gaze direction of the user by processing sensor data from the HMD. The HMD displays a marker overlaid on the electrogram waveform at a location based on an intersection point between the gaze direction and the electrogram waveform. The EGM processing system determines a measurement of the electrogram waveform using the location of the marker. The HMD displays the measurement of the electrogram waveform.

Electrogram annotation system

A sticker system includes at least a head-mounted device and a sticker. In an embodiment, the sticker system receives an image of the sticker captured by an imaging sensor. The sticker system determines registration information by processing a fiducial marker of the sticker shown in the image. The sticker system determines location information of the HMD. Additionally, the sticker system determines a position of the sticker relative to the HMD using the registration information and the location information. The sticker system provides a (e.g., augmented reality) graphic to the HMD for presentation to the user based on the position of the sticker.

Disposable sticker within augmented reality environment

Systems and methods for virtual reality or augmented reality (VR/AR) visualization of 3D medical images using a VR/AR visualization system are disclosed. The VR/AR visualization system includes a computing device operatively coupled to a VR/AR device, and the VR/AR device includes a holographic display and at least one sensor. The holographic display is configured to display a holographic image to an operator. The computing device is configured to receive at least one stored 3D image of a subject's anatomy and at least one real-time 3D position of at least one surgical instrument. The computing device is further configured to register the at least one real-time 3D position of the at least one surgical instrument to correspond to the at least one 3D image of the subject's anatomy, and to generate the holographic image comprising the at least one real-time position of the at least one surgical instrument overlaid on the at least one 3D image of the subject's anatomy.

Growing community of inventors

St. Louis, MO, United States of America

Jonathan R Silva