International Business Machines Corporation

Rami Ben-Ari

Udi Barzelay

Elad Amrani

Tal Hakim

Pavel Kisilev

Daniel Nechemia Rotman

Ran Bakalo

Aviad Zlotnick

Ophir Azulai

Guy Amit

Ayelet Akselrod-Ballin

Tal Tlusty Shapiro

Jacob Goldberger

Yoni Choukroun

Jeremias Sulam

Mor Shpigel Nacson

An example system includes a processor to receive unannotated multimodal data. The processor can estimate a probability an associated pair of different modalities in the unannotated multimodal data to be correctly associated using a multimodal similarity function and a local density estimation. The processor can also train a multimodal representation learning model on the unannotated multimodal data using the estimated probability as a weight for the associated pair in a loss function.

Training multimodal representation learning model on unannotated multimodal data

Approaches presented herein enable action recognition. More specifically, a plurality of video segments having one or more action representations is received. One or more sub-action representations in the plurality of video segments are learned. An embedding in a space of a distance metric learning (DML) network for each of the one or more sub-action representations is determined. A set of respective trajectory distances between each of the one or more sub-action representations and one or more class representatives in the space of the DML network based on the embedding is computed, and the one or more action representations based on the set of respective trajectory distances are classified.

Action recognition using limited data

An example system includes a processor to receive an image containing an object to be detected. The processor is to detect the object in the image via a binary object detector trained via a self-supervised training on raw and unlabeled videos.

Object detector trained via self-supervised training on raw and unlabeled videos

An example system includes a processor to receive raw and unlabeled videos. The processor is to extract speech from the raw and unlabeled videos. The processor is to extract positive frames and negative frames from the raw and unlabeled videos based on the extracted speech for each object to be detected. The processor is to extract region proposals from the positive frames and negative frames. The processor is to extract features based on the extracted region proposals. The processor is to cluster the region proposals and assign a potential score to each cluster. The processor is to train a binary object detector to detect objects based on positive samples randomly selected based on the potential score.

Training an object detector using raw and unlabeled videos and extracted speech

A video scene detection machine learning model is provided. A computer device receives feature vectors corresponding to audio and video components of a video. The computing device provides the feature vectors as input to a trained neural network. The computing device receives from the trained neural network, a plurality of output feature vectors that correspond to shots of the video. The computing device applies optimal sequence grouping to the output feature vectors. The computing device further trains the trained neural network based, at least in part, on the applied optimal sequence grouping.

Deterministic learning video scene detection

An example system includes a processor to receive input data comprising noisy positive data and clean negative data. The processor is to cluster the input data. The processor is to compute a potential score for each cluster of the clustered input data. The processor is to iteratively refine cluster quality of the clusters using the potential scores of the clusters as weights. The processor is to train a classifier by sampling the negative dataset uniformly and the positive set in a non-uniform manner based on the potential score.

Classifier training using noisy samples

There is provided a method for training a deep convolutional neural network (CNN) for detecting an indication of likelihood of abnormality, comprising: receiving anatomical training images, each including an associated annotation indicative of abnormality for the whole image without an indication of location of the abnormality, executing, for each anatomical training image: decomposing the anatomical training image into patches, computing a feature representation of each patch, computing for each patch, according to the feature representation of the patch, a probability that the patch includes an indication of abnormality, setting a probability indicative of likelihood of abnormality in the anatomical image according to the maximal probability value computed for one patch, and training a deep CNN for detecting an indication of likelihood of abnormality in a target anatomical image according to the patches of the anatomical training images, the one patch, and the probability set for each respective anatomical training image.

Systems and methods for automatic detection of an indication of abnormality in an anatomical image

A system for analyzing image data can include a processor to detect a set of medical images and partition each medical image into a plurality of non-overlapping or overlapping patches. The processor can also generate an ordered set of patch clusters derived from the medical images, wherein the patch clusters are sorted based on a type of tissue identified in each of the medical images. Additionally, the processor can generate a digital representation of a percentage of the type of tissue calculated for each of the medical images and transmit an alert in response to detecting that the percentage of the type of tissue for an image exceeds a predetermined threshold.

Medical image data analysis

Embodiments may classify medical images, such as mammograms, using weakly labeled data sets, fully labeled data sets, or a combination of both. For example, a method may comprise receiving a whole medical image, extracting a plurality of image patches from the whole medical image, each image patch including a portion of the whole image, generating a representation of features found in the plurality of image patches, classifying each image patch as including a malignant abnormality, a benign abnormality or not including an abnormality to form a classification for each patch, in parallel, the detection branch computes a malignant distribution over patches and a benign distribution over patches resulting in ranking of patches compare to one another for malignancy, and ranking of patches compare to one another for benign. Patches classification probabilities and ranking are multiplied and summed for malignant and benign, resulting in global malignant probability and global benign probability.

Weakly and fully labeled mammogram classification and localization with a dual branch deep neural network

Embodiments of the present systems and methods may provide the capability to classify medical images, such as mammograms, in an automated manner using existing annotation information. In embodiments, only the global, image level tag may be needed to classify a mammogram into certain types, without fine annotation of the findings in the image. In an embodiment, a computer-implemented method for classifying medical images may comprise receiving a plurality of image tiles, each image tile including a portion of a whole view, processed by a trained or a pre-trained model and outputting a one-dimensional feature vector for each tile to generate a three-dimensional feature volume and classifying the larger image by a trained model based on the generated three-dimensional feature volume to form a classification of the image.

Classifying medical images using deep convolution neural network (CNN) architecture

There is provided a method, comprising: segmenting fibroglandular tissue of a 2D mammographic image of a breast, extracting regions within the segmented fibroglandular tissue and within a boundary portion between the segmented fibroglandular tissue and non-fibroglandular tissue, computing representations for each RoI by a pre-trained deep neural network, training a classifier on the representations to compute a probability score of architectural distortion for each RoI, clustering RoIs defined as positive for architectural distortion using a mean-shift method and providing an indication of the probability of the presence of architectural distortion around a cluster based on the probability distribution of cluster RoI members, removing small clusters having fewer RoI members than a small number threshold, classifying the image as positive for the indication of architectural distortion when at least one cluster remains, or classifying the image as negative for the indication of architectural distortion when no cluster remains.

Systems and methods for automatic detection of architectural distortion in two dimensional mammographic images

Embodiments of the present invention provide automated systems and methods for segmentation of fibro-glandular (FG) tissue in digital mammography. A classifier is trained for breast density without the prior knowledge for FG tissue. The classifier is then used for feature selection, where the selected features are fed into a fuzzy logic module, and an adaptive threshold is obtained. Post-processing is performed on the image in order to reduce regions which may have been misclassified during the FG segmentation.

Growing community of inventors

Kiryat-Ono, Israel

Rami Ben-Ari