International Business Machines Corporation

Vadim Ratner

Yoel Shoshan

Alon Hazan

Aviad Zlotnick

Ella Barkan

Daniel Khapun

Flora Gilboa-Solomon

Yishai Shimoni

Tal El-Hay

Flora Gilboa

Yonatan Keren

A method for improving machine learning algorithm performance is described. The method may comprise receiving a first constituent image of human tissue; receiving a second constituent image of human tissue; overlapping a portion of the second constituent image on a portion of the first constituent image to create an augmented image; and training a model using a dataset comprising at least the augmented image.

Improving model performance by artificial blending of healthy tissue

A method comprising: receiving a primary training set comprising annotated data samples associated with one or more classes and annotated with class labels; constructing an auxiliary training set comprising at least some of the data samples, wherein each of the data samples is assigned at random to one of a set of identification classes, and annotated with an identification label associated with the identification class; at a training stage, train a machine learning model comprising a primary and auxiliary prediction heads, by: (i) training the primary prediction head on the primary training set to predict the class, and (ii) training the auxiliary prediction head on the auxiliary training set to predict the identification class, wherein an output layer of the machine learning model is configured to output a joint prediction which predicts the class label and is invariant to the identification label.

Random classification model head for improved generalization

A method including: receiving, as input, an image; providing a neural network structure including a plurality of multilayer multi-scale neural networks, wherein the plurality of multilayer multi-scale neural networks are arranged sequentially, by laterally connecting corresponding scale-level layers between each two adjoining multilayer multi-scale neural networks in the sequence; and at a training stage, training the neural network structure on a training dataset, to obtain a trained machine learning model configured to perform a computer vision task which includes outputting at least one of: (i) a classification of the image into one class of a set of two or more classes, (ii) a segmentation of a least one object in the image, and (iii) a detection of at least one object in the image.

Unified framework for multigrid neural network architecture

Embodiments may include novel techniques to improve detection of objects in images, for example, in Digital Breast Tomography and that are applicable to ensembles of detectors. For example, a method may comprise generating a plurality of candidate bounding boxes for each of a plurality of image slices of imaged tissue, each generated candidate bounding box having a probability score, collecting at least some of the generated candidate bounding boxes for each slice, generating a heat map of the filtered candidate bounding boxes and filtering the candidate bounding boxes in the heat map based on a first threshold of values in the heat map, performing Non-Maximum Suppression on the heat map filtered candidate bounding boxes, and outputting at least one bounding box based on the probability score.

Pre-NMS filtering using a heat map

Embodiments may provide improved techniques for object detection so as to improve the finding of objects and the accuracy of the boundary predictions using parametric curves defined by multiple control points. For example, in an embodiment, a method may be implemented in a computer system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, the method may comprise receiving an image, extracting from the image a plurality of features related to objects shown in the image, generating, from the extracted features, at least one plurality of points representing a parametric curve bounding an object shown in the image; and outputting the plurality of points representing the parametric curve.

Parametric curves based detector network

Described are techniques for training an image classifier using an augmentation loss function. The techniques including inputting corresponding pairs of a plurality of training images to an image classifier, where respective pairs of the corresponding pairs comprise at least two images having a same classification and different augmentations. The techniques further including training an artificial neural network of the image classifier to classify the plurality of training images using an augmentation loss function, wherein the augmentation loss function reduces differences in model outputs between the corresponding pairs of the plurality of training images.

Augmentation loss function for image classification

An approach for improving object detection performance by using bilateral organ information disclosed. The approach comprises introducing a penalty term to an object localization model. The penalty term encourages the model to identify same physical regions (actual same physical location in the patient's body) in multiple images, which create an image of the same organ. The approach includes a sub-component of the model that can output an entity embedding, in additional to the conventional classification and localization prediction. During optimization process, similar entity embedding for the same logical entity are encouraged, and similar entity embedding for different logical entities are discouraged.

Medical object detection and identification via machine learning

There is provided a method for generating a composite 2D interpretation image, comprising inputting each 2D medical image of a divided 3D image, into a 2D classifier trained on 2D images labelled with an indication of a visual finding, computing a respective explanation map for each respective 2D image, the respective explanation map including regions corresponding to corresponding regions of the respective 2D image, each respective region of the respective explanation map is associated with a computed explainable weight indicative of an influence of the respective corresponding region of the respective 2D image on the outcome of the 2D classifier fed the respective 2D image, and computing a composite 2D interpretation image comprising a respective aggregation weight for each respective region thereof, each respective aggregation weight computed by aggregating the explainable weights computed for the respective regions corresponding to the respective region of the composite 2D interpretation image.

Systems and methods for generating a 2D image from a 3D medical image

An example system includes a processor to receive an input and a model trained to classify inputs. The processor is to iteratively generate a perturbed input that optimizes a saliency metric including a classification term, a sparsity term, and a smoothness term, while keeping parameters of the model constant. The processor is to also detect that a predefined number of iterations is exceeded or a convergence of values of the perturbed input. The processor is to further generate a saliency mask based on a perturbation of the perturbed input in response to detecting the predefined number of iterations is exceeded or the convergence.

Generating saliency masks for inputs of models using saliency metric

There is provided a computer implemented method for detection of likelihood of malignancy in an anatomical image of a patient for planning treatment thereof, comprising: receiving a pair of images that are either of a same patient or from two different patients, wherein the pair of images comprises anatomically corresponding anatomical images each depicting internal anatomical structures of the patient, feeding the pair of images into a model, outputting by the model, an indication of whether the pair of images are of a same patient or not, and generating an indication of likelihood of malignancy when the model wrongly outputs that the pair of images are not of the same patient, when in fact the pair of images are of the same patient, wherein treatment of the patient is planned according to the indication of likelihood of malignancy.

Systems and methods for detecting likelihood of malignancy for a patient using a pair of medical images

Embodiments may exclude portions of input data in order to improve the accuracy and explanatory quality of the output of machine learning models by disregarding parts of the input during the optimization process by masking them during backpropagation. For example, in an embodiment, a method may be implemented in a computer system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, the method may comprise receiving, at the computer system, input data and a machine learning model to generate a prediction based on the input data, generating, at the computer system, a mask indicating portions of the input data to be disregarded during backpropagation of the machine learning model, and modifying, at the computer system, the generated mask to improve the prediction of the machine learning model.

Region constrained regularized adversarial examples for model interpretability

Automated explanation of machine learning predictions, by: Obtaining an input, a machine learning model, and a classification score produced by application of the machine learning model to the input. Optimizing iterative application of the machine learning model to perturbations of the input, wherein a target of said optimizing is at least a predefined non-zero classification score. Generating, based on one of the perturbations which achieved the predefined non-zero classification score, an explanation to the classification score produced by the application of the machine learning model to the input.

Automated explanation of machine learning predictions using classification score targeting

There is provided a method for unsupervised training of a machine learning model, comprising: receiving 3D images depicting a respective object, for each respective 3D image: dividing the 3D image into 3D patches, computing a first 2D image corresponding to a first orientation of the respective object, computing a second 2D image corresponding to a second orientation, automatically labelling pairs of 2D patches from the first and second 2D images with a patch measure indicative of likelihood of a certain 3D patch of the 3D image corresponding to a certain pair of 2D patches, training the ML model using a training dataset including the labelled patch pairs, for receiving patches extracted from first and second 2D images captured by an imaging sensor at the first and second orientations, and outputting an indication of likelihood of a visual finding in a 3D region of the object corresponding to the 2D patches.

Machine learning model for analysis of 2D images depicting a 3D object

Embodiments of the present systems and methods may provide techniques that are applicable to a variety of imaging modalities and that utilize prior knowledge of the dynamics of a physiological system to analytically generate augmented samples for machine learning. For example, in an embodiment, a method implemented in a computer comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, the method may comprise generating a transform for an image of tissue based on deformation of the tissue under compression, obtaining an image of tissue using an imaging modality, and generating an output image by transforming the image of the tissue using the transform.

Methods to induce non-rigid mesh transformation in-variance to CNN based medical imaging classifiers

In an approach for classifying an image containing a continuous characteristic using an image classifier, an image tagged with a single value is received, wherein the single value is from a set of ordered value. A processor sets at least two thresholds based on the set of ordered values. A processor derives at least two labels according to the single value tagged in the image and the at least two thresholds. A processor runs image through an image classifier. A processor receives at least two predictions from the image classifier based on the at least two labels. A processor adjusts the image classifier based on differences between the at least two predictions and the at least two labels.

Growing community of inventors

Haifa, Israel

Vadim Ratner