Hewlett-packard Development Company, L.p.

International Business Machines Corporation

Entit Software LLC

Pavel Kisilev

Suk Hwan Lim

Eugene Walach

Sharon Alpert

Daniel Freedman

Doron Shaked

Carl Staelin

Sagi Schein

Ella Barkan

Boris Oicherman

Renato Keshet

Ruth Bergman

Mani Fischer

Rami Ben-Ari

Anastasia Dubrovina

Miri Erihov

Sharbell Hashoul

Ron Maurer

Asaf Tzadok

Michal Aharon

Hila Nachlieli

Dan S Chevion

Andre Heilper

Gitit Ruckenstein

Ron Banner

Lior Shapira

Gregory Braverman

Yohanan Sivan

Ayelet Akselrod-Ballin

Boaz Ophir

Ran Bakalo

Yoni Choukroun

Alexander Berkovich

Jeremias Sulam

Eli A Meirom

Eliyahu Sason

Presented herein are methods, systems, devices, and computer-readable media for image annotation for medical procedures. The system operates in a parallel manner. In one flow, the system starts from clinical terms and image and applies image detection module in order to get visual candidates for related radiological finding and provide them with semantic descriptors. In the second (parallel) flow, the system produces a list of prioritized semantic descriptors (with probabilities). The second flow is done by application of a reverse inference algorithm that uses clinical terms and expert clinical knowledge. The results of both flows combined by matching module for detection the best candidate and with limited user input images can be annotated. The clinical terms are extracted from clinical documents by textual analysis.

Method for automatic visual annotation of radiological images from patient clinical data

There is provided a method for training a deep convolutional neural network (CNN) for detecting an indication of likelihood of abnormality, comprising: receiving anatomical training images, each including an associated annotation indicative of abnormality for the whole image without an indication of location of the abnormality, executing, for each anatomical training image: decomposing the anatomical training image into patches, computing a feature representation of each patch, computing for each patch, according to the feature representation of the patch, a probability that the patch includes an indication of abnormality, setting a probability indicative of likelihood of abnormality in the anatomical image according to the maximal probability value computed for one patch, and training a deep CNN for detecting an indication of likelihood of abnormality in a target anatomical image according to the patches of the anatomical training images, the one patch, and the probability set for each respective anatomical training image.

Systems and methods for automatic detection of an indication of abnormality in an anatomical image

Asymmetries are detected in one or more images by partitioning each image to create a set of patches. Salient patches are identified, and an independent displacement for each patch is identified. The techniques used to identify the salient patches and the displacement for each patch are combined in a function to generate a score for each patch. The scores can be used to identify possible asymmetries.

Unsupervised asymmetry detection

Embodiments of the present systems and methods may provide the capability to classify medical images, such as mammograms, in an automated manner using existing annotation information. In embodiments, only the global, image level tag may be needed to classify a mammogram into certain types, without fine annotation of the findings in the image. In an embodiment, a computer-implemented method for classifying medical images may comprise receiving a plurality of image tiles, each image tile including a portion of a whole view, processed by a trained or a pre-trained model and outputting a one-dimensional feature vector for each tile to generate a three-dimensional feature volume and classifying the larger image by a trained model based on the generated three-dimensional feature volume to form a classification of the image.

Classifying medical images using deep convolution neural network (CNN) architecture

A computer-implemented method includes receiving multimodal data. The computer-implemented method further includes generating one or more kernel matrices from the multimodal data. The computer-implemented method further includes generating an equivalent kernel matrix using one or more coefficient matrices, wherein the one or more coefficient matrices are constrained by a nuclear norm. The computer-implemented method further includes initiating one or more iterative processes. Each of the one or more iterative processes includes: calculating an error for the one or more coefficient matrices of the equivalent kernel matrix based on a training set, and initiating a line search for the one or more coefficient matrices of the equivalent kernel matrix. The computer-implemented method further includes, responsive to generating an optimal coefficient matrix, terminating the one or more iterative processes. The method may be embodied in a corresponding computer system or computer program product.

Enhanced kernel representation for processing multimodal data

A computer implemented method of automatically creating a classification function trained with augmented representation of features extracted from a plurality of sample media objects using one or more hardware processors for executing a code. The code comprises code instructions for extracting a plurality of features from a plurality of sample media objects, generating a plurality of feature samples for each of the plurality of features by augmenting the plurality of features, training a classification function with the plurality of features samples and outputting the classification function for classifying one or more new media objects.

Creating deep learning models using feature augmentation

In one implementation, a plurality of signature vectors from a multi-dimensional representation of a graphical object is generated. Each of the signature vectors comprises attributes that vary little in response to changes in shape, size, orientation, and visual layer appearance of the graphical object, and each of the signature vectors includes attributes based on operations of integration, differentiation, and transforms on the multi-dimensional representation of the graphical object. Each signature vector is composited into multiple portions from the plurality of signature vectors to define an appearance-invariant signature of the graphical object.

Graphical object appearance-invariant signature

An example system includes a processor to train a convolutional neural network (CNN) to detect features, and train fully connected layers of the CNN to map detected features to semantic descriptors, based on a data set including one or more lesions. The processor is to also receive a medical image to be analyzed for lesions. The processor is to further extract feature maps from the medical image using the trained CNN. The processor is also to detect a region of interest via the trained CNN and generate a bounding box around the detected region of interest. The processor is to reduce a dimension of the region of interest based on the feature maps. The processor is to generate a semantic description of the region of interest via the trained fully connected layers.

Automatic detection and semantic description of lesions using a convolutional neural network

A computer implemented method, a computerized system and a computer program product for generating questions. The computer implemented method comprising obtaining a question, wherein the question comprises one or more elements that define an answer for the question. The method further comprising obtaining the answer. The method further comprises automatically generating, by a processor, a new question based on the question and the answer. The automatic generation comprises determining a variant of the one or more elements, wherein the variant defines the answer, wherein the new question comprises the variant.

Question generator based on elements of an existing question

A computer implemented method, a computerized system and a computer program product for automatic image segmentation. The computer implemented method comprises obtaining an image of a tissue, wherein the image is produced using an imaging modality. The method further comprises automatically identifying, by a processor, a tissue segment within the image, wherein said identifying comprises identifying an artifact within the image, wherein the artifact is a misrepresentation of a tissue structure, wherein the misrepresentation is associated with the imaging modality; and searching for the tissue segment in a location adjacent to the artifact.

Automatic image segmentation

A method comprising using at least one hardware processor for computing a patch distinctiveness score for each of multiple patches of a medical image, computing a shape distinctiveness score for each of multiple regions of the medical image, and computing a saliency map of the medical image, by combining the patch distinctiveness score and the shape distinctiveness score.

Anomaly detection in medical imagery

A method comprising using at least one hardware processor for applying a mapping function to a medical image, to generate a semantic description of a visual finding in the medical image. The mapping function is optionally an MRF (Markov random field)-based, SVM (Support Vector Machine) mapping function.

Automatic generation of semantic description of visual findings in medical images

An image is processed in a device-independent color space by applying a modification function to one or more dimensions of that color space. The image is then converted into a target device-dependent color space. In order to minimize unwanted changes of color arising from the modification function taking image color values out of gamut when converted to the target device-dependent color space, the modification function is scaled to limit modified image values to within modification limits that are determined, for each image pixel or group of pixels, in dependence on the target device-dependent color space gamut boundary in the device-independent color space.

Minimizing unwanted changes of color during image processing

Growing community of inventors

Maalot, Israel

Pavel Kisilev