International Business Machines Corporation

Guerbet

Dana Levanony

Efrat Hexter

Giovanni John Jacques Palma

Yi-Qing Wang

Moshe Raboh

Arkadiusz Sitek

Paul Dufort

Omid Bonakdar Sakhi

Tal Tlusty Shapiro

Pedro Luis Esquinas Fernandez

Thomas Binder

Shaked Perek

Aviad Zlotnick

Flora Gilboa-Solomon

Simona Rabinovici-Cohen

Tal El-Hay

Ran Bakalo

Ami Abutbul

According to one embodiment, a method, computer system, and computer program product for hard negative training is provided. The embodiment may include a computer receiving a training set, where the training set comprises one or more training samples. The computer trains a deep neural network (DNN) with the training set. The embodiment may also include determining, using the DNN, information for each of the one or more training samples, where the information includes one or more scores associated with the one or more training samples. The embodiment may further include generating a training epoch from the one or more training samples based on the information and updates the information based on using the training epoch with the DNN.

Estimated online hard negative mining via probabilistic selection and scores history consideration

A method and system perform single phase and multi-phase contour refinement of lesions. The method includes receiving a three dimensional input mask; receiving input slices from the medical images including a lesion; cropping the input slices with the input mask; performing lesion contour refinement for the cropped input slices and the input mask to obtain a predicted mask; and storing the predicted mask that includes 3D lesion contour refinement. A multiphase method includes deforming the 3D input mask from the reference phase to a target phase or warping the input slices from the target phase to the reference phase before contour refinement. The warped images generate an output mask in the reference phase coordinate system that is then deformed to the target phase coordinate system for display.

Multi-phase object contour refinement

An example system includes a processor to receive an image with corresponding acquisition information. The processor is to classify the image using the corresponding acquisition information via a deep neural network including integrated acquisition information.

Classifying images using deep neural network with integrated acquisition information

A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. First ML model(s) process an input volume of medical images (VOI) to determine whether VOI depicts a predetermined amount of an anatomical structure. The AI pipeline determines whether criteria, such as a predetermined amount of an anatomical structure of interest being depicted in the input volume, are satisfied by output of the first ML model(s). If so, lesion processing operations are performed including: second ML model(s) processing the VOI to detect lesions which correspond to the anatomical structure of interest; third ML model(s) performing lesion segmentation and combining of lesion contours associated with a same lesion; and fourth ML models processing the listing of lesions to classify the lesions. The AI pipeline outputs the listing of lesions and the classifications for downstream computing system processing.

Lesion detection artificial intelligence pipeline computing system

Embodiments herein disclose computer-implemented methods, computer program products and computer systems for annotating magnetic resonance imaging (MRI) images. The method may include receiving mammogram (MG) image data representing annotated MG images of a patient breast, the annotated MG images being one of either a craniocaudal view or of a mediolateral oblique view. The method may include identifying annotations representing an abnormality at a first location in the annotated MG images; receiving MRI image data representing MRI images of the patient breast; generating annotated MRI image data using the MRI image data and the annotations identified in the annotated MG images, the annotated MRI image data including MRI annotations at a second location based at least in part on the first location, the MRI annotations in the annotated MRI image data representing the abnormality; and storing the annotated MRI image data in a database.

MRI annotation

Techniques for training machine learning models for improved accuracy at classifying medical imaging data sets by trimming ambiguous samples from training data sets are described herein. In some embodiments, a machine learning model is trained using a data set, where a subset of the data set comprises data with a conflict between a first label based on an expert opinion and a second label based on a ground truth based on a medical examination. During some epochs of training the machine learning model, loss values for each data sample in the epoch are compared against a loss threshold, with data samples with corresponding loss values that exceed the loss threshold that also belong to the subclass trimmed from the data set for subsequent epochs of training. The loss threshold for the next epoch is then updated based on loss values of the trimmed data set.

Training machine learning models to exclude ambiguous data samples

An example system includes a processor to evaluate a trained first classifier on a test set of labeled data to generate error rates for a number of labels. The processor is to process a set of unlabeled data via the trained first classifier to generate annotated data including labels and associated error rates. The processor is to train a second classifier using the annotated data and the associated error rates.

Annotating unlabeled data using classifier error rates

A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. First ML model(s) process an input volume of medical images (VOI) to determine whether VOI depicts a predetermined amount of an anatomical structure. The AI pipeline determines whether criteria, such as a predetermined amount of an anatomical structure of interest being depicted in the input volume, are satisfied by output of the first ML model(s). If so, lesion processing operations are performed including: second ML modal(s) processing the VOI to detect lesions which correspond to the anatomical structure of interest; third ML model(s) performing lesion segmentation and combining of lesion contours associated with a same lesion; and fourth ML models processing the listing of lesions to classify the lesions. The AI pipeline outputs the listing of lesions and the classifications for downstream computing system processing.

An example system includes a processor to receive a three-dimensional (3D) volume. The processor can partition the 3D volume into slices. The processor can generate, via a two-dimensional (2D) neural network, slice features based on the slices. The processor can generate, via a three-dimensional (3D) neural network, a three-dimensional (3D) feature volume based on the slice features. The processor can generate, via a volume predictor, a volume prediction based on the 3D feature volume.

Generating volume predictions of three-dimensional volumes using slice features

A method including: automatically detecting, using at least one machine learning algorithm, one or more abnormalities depicted in a medical image of a patient; automatically determining whether the one or more abnormalities have remained temporally and unchanged, based on an older medical image of the patient; and upon determining that the one or more abnormalities have remained temporally and spatially unchanged: automatically inpainting the one or more abnormalities in the medical image, and automatically enrich a new training set with the inpainted medical image.

Training set enrichment with insignificantly-abnormal medical images

Embodiments may include techniques to choose a model based on a similarity of computed features of an input to computed features of several models in order to improve feature analysis using Machine Learning models. A method of image analysis may comprise extracting a training feature vector corresponding to each of the plurality of machine learning models from each validation image from a plurality of machine learning models trained using a plurality of validation images, extracting from a new image a new feature vector corresponding to each of the plurality of machine learning models, comparing each new feature vector corresponding to each machine learning model with the training feature vector corresponding to each of the plurality of machine learning models, and selecting and outputting an inference for the new image generated by the machine learning model for which the new feature vector and the training feature vector are most similar.

Similarity based per item model selection for medical imaging

A method which includes: Obtaining a training set which comprises: multiple data pairs each comprising: (i) a raw sensory signal acquired by a medical imaging system, and (ii) a processed image generated by the medical imaging system from the raw sensory signal; and a classification label for each of the data pairs. Based on the training set, training an artificial neural network (ANN), wherein the training comprises minimizing a global loss which is a weighted sum of: a loss between the classification labels and classification predictions by the ANN, and a similarity loss between the processed images and images generated by an intermediate layer of the ANN. The training is such that the trained ANN is configured, for a new raw sensory signal: to predict a new classification, and to generate a new image by the intermediate layer of the ANN.

Growing community of inventors

Tel Aviv, Israel

Dana Levanony