International Business Machines Corporation

Merative US L.p.

Alexandros Karargyris

Thomas Guthrie Zimmerman

Tanveer Fathima Syeda-Mahmood

Hongzhi Wang

Mehdi Moradi

Joy Tzung-yu Wu

Yaniv Gur

Chun Lok Wong

Satyananda Kashyap

Ahmed El Harouni

Anup Pillai

Ashutosh Jadhav

Joy Wu

Methods and systems are directed to training an artificial intelligence engine. One system includes an electronic processor configured obtain a set of reports corresponding to a set of medical images, determine a label for a finding of interest, and identify one or more ambiguous reports in the set of repots. Ambiguous reports do not include a positive label or a negative label for the finding of interest. The electronic processor is also configured to generate an annotation for each of the one or more ambiguous reports in the set of reports, and train the artificial intelligence engine using a training set including the annotation for each of the one or more ambiguous reports and non-ambiguous reports in the set of reports. A result of the training is generation of a classification model for the label for the finding of interest.

Rule out accuracy for detecting findings of interest in images

An ophthalmic device having a single or dual compartment configuration selectively emits infrared and visible light beams onto one or a pair of target eyes. The device performs eye fundus imaging and aids in the detection of ailments as indicated by anomalies in the pupillary reflex.

Ophthalmoscope using natural pupil dilation

Embodiments of the present disclosure include a computer-implemented method, a system, and a computer program product for integrating bottom-up segmentation techniques into a semi-supervised image segmentation machine learning model. The computer implemented method includes training a machine learning model with a labeled dataset. The labeled dataset includes ground truth segmentation labels for each sample in the labeled dataset. The computer implemented method also includes generating a pseudo labeled dataset by applying an unlabeled dataset to the machine learning model using a top-down segmentation grouping rule. The computer implemented method further includes evaluating the pseudo labeled dataset using a bottom-up segmentation grouping rule to produce evaluation results, combining the pseudo labeled dataset with the second pseudo labeled dataset into a training dataset, and then retraining the machine learning model with the training dataset.

Integrated bottom-up segmentation for semi-supervised image segmentation

Mechanisms are provided for automatically annotating input images with bounding region annotations and corresponding anomaly labels. The mechanisms segment an input image to generate a mask corresponding to recognized internal structures of a subject. A template data structure is generated that specifies standardized internal structure zones of the subject. The mechanisms register the mask with the template data structure to generate a template registered mask identifying standardized internal structure zones present within the mask, and generate bounding region annotations for each standardized internal structure zone of the template registered mask. The bounding region annotations are correlated with labels indicating whether or not the bounding region comprises an anomaly in the input image based on an analysis of a received natural language text description of the input image. The bounding region annotations and labels are stored in association with the input image.

Automatic bounding region annotation for localization of abnormalities

Mechanisms are provided to implement an automated medical imaging report generator which receives an input medical image and inputs the input medical image into a machine learning (ML) computer model trained to predict finding labels based on patterns of image features extracted from the medical image. The ML computer model generates a prediction of a finding label applicable to the input medical image in terms of a finding label prediction output vector. Based on the finding label prediction output vector, a lookup operation is performed, in a medical report database of previously processed medical imaging report data structures, to find a matching medical imaging report data structure corresponding to the finding label. An output medical imaging report is generated for the input medical image based on natural language content of the matching medical imaging report data structure.

Automatic generation of medical imaging reports based on fine grained finding labels

Methods and systems for determining a diagnostically unacceptable medical image. One system includes at least one electronic processor configured to receive a new medical image captured via a medical imaging device. The at least one electronic processor is also configured to determine a classification of the new medical image using a model developed with machine learning using training information that includes a plurality of medical images and an associated classification for each medical image, each associated classification identifying whether the associated medical image is diagnostically unacceptable, wherein the classification of the new medical image indicates whether the new medical image is diagnostically unacceptable. The at least one electronic processor is also configured to, when the classification indicates that the new medical image is diagnostically unacceptable, prompt a user of the medical imaging device to adjust a parameter associated with the new medical image and recapture the new medical image.

Methods and systems for determining a diagnostically unacceptable medical image

Example-based affine registration is provided. In various embodiments, a plurality of training images is read. A predetermined affine transform is read for each of the plurality of training images. Each affine transform maps its associated image to a template. Weights are determined for each of the plurality of training images. The weights are determined to minimize a difference between the test image and a weighted linear combination of the training images. An affine transform is determined mapping the test image to the template by computing a weighted linear combination of the affine transforms using the weights.

Techniques for example-based affine registration

A mechanism is provided in a data processing system comprising a processor and a memory, the memory comprising instructions that are executed by the processor to specifically configure the processor to implement a multi-modal classification and segmentation engine for anatomical segmentation identifying modes and viewpoints in biomedical images. The mechanism trains a neural network perform simultaneous classification and segmentation using a set of training images. The neural network provides a classification output that identifies a class label and a second output that identifies a segmentation label. The multi-modal classification and segmentation engine provides a biomedical image as the input image to the neural network. The neural network outputs a plurality of class label probabilities for a plurality of class labels and a plurality of segmentation label probabilities for each of a plurality of segmentation labels. A post-processing component executing within the multi-modal classification and segmentation engine classifies the biomedical image as an identified modality and an identified viewpoint based on the plurality of class label probabilities. The multi-modal classification and segmentation engine segments the biomedical image based on the plurality of segmentation label probabilities. The multi-modal classification and segmentation engine outputs the classified and segmented biomedical image.

Growing community of inventors

San Jose, CA, United States of America

Alexandros Karargyris