International Business Machines Corporation

Guerbet

Merative US L.p.

Ehsan Dehghan Marvast

Tanveer Fathima Syeda-Mahmood

David James Beymer

Allen Lu

Mehdi Moradi

Amin Katouzian

Ahmed El Harouni

Anup Pillai

Ashutosh Jadhav

Satyananda Kashyap

Chaitanya Shivade

Vandana Mukherjee

Marina Bendersky

Yaniv Gur

Hongzhi Wang

Deepta Rajan

Luyao Shi

Nathaniel Mason Braman

Shafiqul Abedin

Girish Narayan

A candidate generator generates a set of candidate three-dimensional image patches from an input volume. A candidate classifier classifies the set of candidate three-dimensional image patches as containing or not containing disease. Classifying the set of candidate three-dimensional image patches comprises generating an attention mask for each given candidate three-dimensional image patch within the set of candidate three-dimensional image patches to form a set of attention masks, applying the set of attention masks to the set of candidate three-dimensional image patches to form a set of masked image patches, and classifying the set of masked image patches as containing or not containing the disease. The candidate classifier applies soft attention and hard attention to the three-dimensional image patches such that distinctive image regions are highlighted proportionally to their contribution to classification while completely removing image regions that may cause confusion.

Disease detection with masked attention

Methods and systems for training computer-aided condition detection systems. One method includes receiving a plurality of images for a plurality of patients, some of the images including an annotation associated with a condition; iteratively applying a first deep learning network to each of the images to produce an attention map, a feature map, and an image-level probability of the condition for each of the images; iteratively applying a second deep learning network to each feature map produced by the first network to produce a plurality of outputs; training the first network based on the attention map produced for each image; and training the second network based on the output produced for each of the patients. The second network includes a plurality of convolution layers and a plurality of convolutional long short-term memory (LSTM) layers. Each of the outputs includes a patient-level probability of the condition for one of the patients.

Automatic condition diagnosis using an attention-guided framework

Methods and systems for training computer-aided condition detection systems. One method includes receiving a plurality of images for a plurality of patients, some of the images including an annotation associated with a condition; iteratively applying a first deep learning network to each of the images to produce a segmentation map, a feature map, and an image-level probability of the condition for each of the images; iteratively applying a second deep learning network to each feature map produced by the first network to produce a plurality of outputs; training the first network based on the segmentation map produced for each image; and training the second network based on the output produced for each of the patients. The second network includes a plurality of convolution layers and a plurality of convolutional long short-term memory (LSTM) layers. Each of the outputs includes a patient-level probability of the condition for one of the patients.

Automatic condition diagnosis using a segmentation-guided framework

Mechanisms are provided to implement an automated echocardiograph measurement extraction system. The automated echocardiograph measurement extraction system receives medical imaging data comprising one or more medical images and inputs the one or more medical images into a deep learning network. The deep learning network automatically processes the one or more medical images to generate an extracted echocardiograph measurement vector output comprising one or more values for echocardiograph measurements extracted from the one or more medical images. The deep learning network outputs the extracted echocardiograph measurement vector output to a medical image viewer.

Automated extraction of echocardiograph measurements from medical images

Mechanisms are provided to implement an automated medical image processing pipeline selection (MIPPS) system. The MIPPS system receives medical image data associated with a patient electronic medical record and analyzes the medical image data to extract evidence data comprising characteristics of one or more medical images in the medical image data indicative of a medical image processing pipeline to select for processing the one or more medical images. The evidence data is provided to a machine learning model of the MIPPS system which selects a medical image processing pipeline based on a machine learning based analysis of the evidence data. The selected medical image processing pipeline processes the medical image data to generate a results output.

Determining appropriate medical image processing pipeline based on machine learning

Systems and methods for developing a disease detection model. One method includes training the model using an image study and an associated disease label mined from a radiology report. The image study including a sequence of a plurality of two-dimensional slices of a three-dimensional image volume, and the model including a convolutional neural network layer and a convolutional long short-term memory layer. Training the model includes individually extracting a set of features from each of the plurality of two-dimensional slices using the convolutional neural network layer, sequentially processing the features extracted by the convolutional neural network layer for each of the plurality of two-dimensional slices using the convolutional long short-term memory layer, processing output from the convolutional long short-term memory layer for each of the plurality of two-dimensional slices to generate a probability of the disease, and updating the model based on comparing the probability to the label.

Disease detection from weakly annotated volumetric medical images using convolutional long short-term memory and multiple instance learning

Systems and methods for developing a disease detection model. One method includes training the model using an image study and an associated disease label mined from a radiology report. The image study including a sequence of a plurality of two-dimensional slices of a three-dimensional image volume, and the model including a convolutional neural network layer and a convolutional long short-term memory layer. Training the model includes individually extracting a set of features from each of the plurality of two-dimensional slices using the convolutional neural network layer, sequentially processing the features extracted by the convolutional neural network layer for each of the plurality of two-dimensional slices using the convolutional long short-term memory layer, processing output from the convolutional long short-term memory layer for a sequentially last of the plurality of two-dimensional slices to generate a probability of the disease, and updating the model based on comparing the probability to the label.

Disease detection from weakly annotated volumetric medical images using convolutional long short-term memory

A mechanism is provided in a data processing system comprising a processor and a memory, the memory comprising instructions that are executed by the processor to specifically configure the processor to implement an intelligent medical image viewing engine. The intelligent medical image viewing engine receives a medical imaging study data structure comprising a plurality of electronic medical images from a medical image database. An image processing component executing within the intelligent medical image viewing engine analyzes the medical imaging study data structure to identify, for each electronic medical image in the plurality of electronic medical images, a corresponding set of image attributes. The intelligent medical image viewing engine receives a user input specifying at least one filter attribute for generating a medical image output and correlates the at least one filter attribute with at least one medical attribute of medical images to be used for selection of electronic medical images from the medical imaging study data structure. An image selection component executing within the intelligent medical image viewing engine selects a subset of the electronic medical images in the plurality of electronic medical images based on the correlation of the at least one filter attribute with the at least one medical attribute. A user interface generation component executing within the intelligent medical image viewing engine generates and outputs a medical image output comprising the subset of electronic medical images based on the selection.

Intelligently organizing displays of medical imaging content for rapid browsing and report creation

Mechanisms are provided for evaluating the normality of a medical condition of a patient based on a medical image. A medical image segmentation receives a medical image and segments the medical image to generate an extracted contour representing an anatomical feature. The medical image segmentation engine correlates the extracted contour with a template shape corresponding to the anatomical feature. A feature extraction engine extracts one or more features from a region of the medical image corresponding to the template shape. A normality classification engine performs a normality classification operation on the extracted one or more features to generate a normality score for the medical image and outputs the normality score to a computing device.

Automated normality scoring of echocardiograms

Medical imaging study summary engine mechanisms are provided. The mechanisms receive a medical imaging study having data representing a plurality of medical images of a patient. The mechanisms generate a temporal trajectory data structure of at least a subset of the medical images in the plurality of medical images, wherein the temporal trajectory data structure specifies topological changes in temporally subsequent medical images in the plurality of medical images. The mechanisms select medical image data corresponding to selected medical images from the medical imaging study data structure based on the temporal trajectory data structure. The mechanisms output the selected medical image data via a medical imaging study user interface.

Automatic summarization of medical imaging studies

A method for processing medical images includes analyzing a medical image to detect a medical condition from a list of medical conditions, wherein the list of medical conditions includes aortic dissection, pulmonary embolism, and coronary stenosis. Responsive to determining the medical image includes a first medical condition, the method generates a first report that includes information on a detection of the first medical condition. The method identifies, a medical specialist based on availability and medical expertise and sends to the identified medical specialist, the medical image and the first report for a decision on the detection of the first medical condition. Responsive to receiving the decision from the medical specialist, the method sends to a second electronic device, the decision, the medical image, and the first report.

Growing community of inventors

Palo Alto, CA, United States of America

Ehsan Dehghan Marvast