Other

Bfly Operations, Inc.

Google Inc.

Butterfly Network, Inc.

Microsoft Technology Licensing, LLC

Nathan Silberman

Gabriel Fine

Cristina Shin

Matthew De Jonge

Igor Lovchinsky

Audrey Howell

David Elgena

Tomer Gafner

Dilip Krishnan

Dumitru Erhan

Konstantinos Bousmalis

Yang Liu

George Trigeorgis

Swaminathan Sankaranarayanan

Jonathan Marc Rothberg

Eyal Ofek

Tyler S Ralston

Pushmeet Kohli

Abraham Neben

Lior Shapira

Ran Gal

Vineet Shah

David Martin Dohan

Ardavan Saeedi

Daniel Hertz

Elizabeth Lauer

David Cairns

Kurt Guenther

A system, method, and computer program product for presurgical planning related to an object associated with a medical procedure for a patient is disclosed. A model can be trained for an object as part of a medical procedure, the trained model being generated from one or more machine learning algorithms that are trained on annotated images of the object with spatial information of the object. An imaging computer system can receive or simulate one or more images of the object captured by an imaging device positioned external to the patient. The imaging computer system can further determine, based on applying the trained model to the one or more images of the object, spatial information of the object during the medical procedure. A display can output the one or more images and the spatial information of the object.

Presurgical planning

A system, method, and computer program product for image-based detection of an object internal to a patient is disclosed. A model can be trained for an internal object, such as an invasive medical device, the trained model being generated from one or more machine learning algorithms that are trained on annotated images of the object with spatial information of the object. An imaging computer system can receive one or more images of the internal object captured by an imaging device positioned external to the patient. The imaging computer system can further detect, based on applying the trained model to the one or more images of the object, the internal object within the patient. A display can output the one or more images and the identifying information for the object.

Image-based detection of internal object

A system, method, and computer program product for image-based detection of conditions for an object internal to a patient is disclosed. A model can be trained for an internal object, such as an invasive medical device, the trained model being generated from one or more machine learning algorithms that are trained on annotated images of the object with condition information of the object. An imaging computer system can receive one or more images of the internal object captured by an imaging device positioned external to the patient. The imaging computer system can further determine, based on applying the trained model to the one or more images of the object, current condition information for the internal object, such as current spatial information for the object. A display can output the one or more images and the condition information for the object.

Image-based detection of object condition

A system and method for image-based monitoring of an object inserted into a patient is disclosed. A model can be trained for an object configured to be inserted into a patient as part of a medical procedure, the trained model being generated from one or more machine learning algorithms that are trained on annotated images of the object with spatial information of the object. An imaging computer system can receive one or more images of the object inserted within the patient captured by an imaging device positioned external to the patient. The imaging computer system can further determine, based on applying the trained model to the one or more images of the object, current spatial information of the object within the patient. A display can output the one or more images and the current spatial information of the object.

System for monitoring object inserted into patient's body via image processing

Aspects of the technology described herein relate to displaying indications of anatomical regions that have been imaged and/or that should be imaged next. In some embodiments, ultrasound data collected from a subject by an ultrasound device may be received, an automatic determination may be made that the ultrasound data was collected from a particular anatomical location, and an indication of the anatomical location may be displayed. In some embodiments, a plurality of anatomical locations may be determined for imaging, an anatomical location from among the plurality of anatomical locations may be automatically selected, and an indication of the anatomical location may be displayed. Displaying an indication of an anatomical location may include displaying or modifying a marker on a frame of a video of the subject such that the marker appears in the frame of the video to be located at the anatomical location on the subject.

Methods and apparatuses for guiding collection of ultrasound data

Aspects of the technology described herein relate to determining, by a processing device in operative communication with an ultrasound device, a position and/or orientation of the ultrasound device relative to the processing device, and displaying on a display screen of the processing device, based on the position and/or orientation of the ultrasound device relative to the processing device, an instruction for moving the ultrasound device.

Methods and apparatuses for ultrasound data collection

Some aspects of the technology described herein relate to configuring an ultrasound device to perform a three-dimensional ultrasound imaging sweep, and displaying ultrasound images and segmented portions of the ultrasound images as the ultrasound images are collected during the three-dimensional ultrasound imaging sweep. Certain aspects relate to displaying an ultrasound image collected by an ultrasound device and configuring the ultrasound device to perform a three-dimensional ultrasound imaging sweep based on the ultrasound image collected by the ultrasound device.

Methods and apparatuses for collection and visualization of ultrasound data

Aspects of the technology described herein relate to automatically calculating and displaying a prediction of a collective opinion of a group of individuals regarding imaging data and/or an output based on the imaging data. In some embodiments, the prediction may be a prediction of the collective opinion of a group of individuals regarding the usability of imaging data, regarding a segmentation of an image, or regarding a measurement performed based on the imaging data.

Methods and apparatuses for analyzing imaging data

Aspects of the technology described herein include determining, during ultrasound imaging, that an anatomical region is clipped by a field of view of an ultrasound image, and providing a notification, during the ultrasound imaging, that the anatomical region is clipped by the field of view of the ultrasound image. Aspects of the technology described herein also include determining that an anatomical region is clipped by a field of view of at least one ultrasound image collected during a three-dimensional ultrasound imaging sweep, and providing a notification that the anatomical region is clipped by the field of view of the at least one ultrasound image collected during the three-dimensional ultrasound imaging sweep.

Methods and apparatuses for collection of ultrasound images

A system and method is disclosed for displaying augmented image data for invasive medical devices. A current orientation and a current position of the invasive medical device within a patient can be determined by applying a trained model of the invasive medical device to unannotated images of the invasive medical device as captured by an imaging device. The images of the invasive medical device can be displayed and overlaid with the current orientation and current position of the invasive medical device. User input can be received to initialize tracking of an orientation and a position of the invasive medical device as the invasive medical device is moved within the patient.

Displaying augmented image data for medically invasive devices via image processing

A system and method is disclosed for augmenting image data of an invasive medical device using optical imaging. An optical imaging sensor, separate from the invasive medical device, can generate images of the medical device within a patient. A trained model for the invasive medical device can be trained on annotated images of the invasive medical device with orientation and distance information of the invasive medical device. An imaging computer system can apply the trained model to images of the invasive medical device within the patient to determine a current orientation and a current distance of the invasive medical device. The images of the invasive medical device as captured by the optical imaging sensor, visual orientation information representing the current orientation of the invasive medical device, and visual distance information representing the current distance of the invasive medical device within the patient can be displayed.

System for augmenting image data of medically invasive devices using optical imaging

A system and method for augmenting unlabeled imaging data of invasive medical devices is disclosed. An imaging device can generate images of the invasive medical device inside a patient. A trained model for the invasive medical device can be trained on annotated images of the invasive medical device having labels indicating orientation and position information. An imaging computer system can apply the trained model to unlabeled images of the invasive medical device within the patient to determine a current orientation and a current position of the invasive medical device inside the patient. The images of the invasive medical device, visual orientation information representing the current orientation of the invasive medical device, and visual position information representing the current position of the invasive medical device inside the patient can be outputted to a display.

Augmenting unlabeled images of medically invasive devices via image processing

A system and method is disclosed for augmenting image data of an invasive medical device having non-medical structures. An invasive medical device having markers with distinct physical shapes relative to other portions of the invasive medical device can be inserted in a patient. An imaging device can generate images of the invasive medical device within the patient. A trained model for the invasive medical device can be trained on annotated images of the invasive medical device annotated with marker and spatial information. An imaging computer system can apply the trained model to images of the invasive medical device having depictions of the markers to determine current spatial information. The depictions of the markers can be identified and correlated with the spatial information from the annotated images. The images and visual spatial information representing the spatial information of the invasive medical device can be outputted to a display.

Augmenting image data of medically invasive devices having non-medical structures

A system and method is disclosed for guiding an invasive medical device with radiation absorbing markers. The invasive medical device can include markers with different radiation absorbing properties relative to other portions of the invasive medical device. An imaging device can generate images of the invasive medical device within a patient. A trained model for the invasive medical device can be trained on annotated images of the invasive medical device annotated with marker information identifying the markers and spatial information for the invasive medical device. An imaging computer system can apply the trained model to images of the invasive medical device within the patient including depictions of the markers to determine current spatial information of the invasive medical device inside the patient. The images of the invasive medical device and visual spatial information representing the spatial information of the invasive medical device can be outputted to a display.

Guiding medically invasive devices with radiation absorbing markers via image processing

A system and method is disclosed for guiding invasive medical devices relative to anatomical structures. An imaging device can generate one or more images of the invasive medical device within the patient. A trained model for the invasive medical device can be trained on annotated images of the invasive medical device with at least one of orientation and position information. An imaging computer system can apply the trained model to unannotated images of the invasive medical device within the patient to determine at least one of a current orientation and a current position of the invasive medical device relative to the one or more anatomical structures within the patient. The images of the invasive medical device, visual orientation information representing the current orientation, and visual position information representing the current position of the invasive medical device relative to the anatomical structures within the patient can be outputted to a display.

Growing community of inventors

Brooklyn, NY, United States of America

Nathan Silberman