International Business Machines Corporation

Pallab K Roy

Rahil Garnavi

Suman Sedai

Dwarikanath Mahapatra

Rajib Chakravorty

Ruwan B Tennakoon

Seyedbehzad Bozorgtabar

Stephen Michael Moore

Kerry J Halupka

A learning sub-system models search patterns of multiple experts in analyzing an image using a recurrent neural network (RNN) architecture, creates a knowledge base that models expert knowledge. A teaching sub-system teaches the search pattern captured by the RNN model and presents to a learning user the information for analyzing an image. The teaching sub-system determines the teaching image sequence based on a difficulty level identified using image features, audio cues, expert confidence and time taken by experts. An evaluation sub-system measures the learning user's performance in terms of search strategy that is evaluated against the RNN model and provides feedback on overall sequence followed by the learning user and time spent by the learning user on each region in the image.

System and method to teach and evaluate image grading performance using prior learned expert knowledge base

A computer-implemented method includes obtaining an image of a retinal fundus. A plurality of features is extracted from the image of the retinal fundus. The plurality of features includes at least one feature based on anatomical domain knowledge of the retinal fundus and at least one response of a pre-trained deep convolutional neural network to at least a portion of the image of the retinal fundus. The retinal fundus is determined to belong to a left eye or a right eye, based on an analysis of the plurality of features.

Automatically detecting eye type in retinal fundus images

An embodiment of the invention receives by an interface a retinal image from a patient, and identifies by a feature extraction device vessel fragments in the retinal image. The vessel fragments include at least a portion of a major vessel and at least a portion of a branch connected to a major vessel. A processor computes estimated blood flow velocities in the vessel fragments with a blood flow velocity estimation model and determines actual blood flow velocities in the vessel fragments. An analysis engine compares the actual blood flow velocities in the vessel fragments to the estimated blood flow velocities in the vessel fragments. The analysis engine detects a candidate plaque affected vessel fragment when the estimated blood flow velocities in the vessel fragments differs from the actual blood flow velocities in the vessel fragments by a predetermined amount.

Early prediction of hypertensive retinopathy

Jointly determining image segmentation and characterization. A computer-generated image of an organ may be received. Organ characteristics estimation may be performed to predict the organ characteristics considering organ segmentation. Organ segmentation may be performed to delineate the organ in the image considering the organ characteristics. A feedback loop feeds the organ characteristics estimation to determine the organ segmentation, and feeds back the organ segmentation to determine the organ characteristics estimation.

Joint segmentation and characteristics estimation in medical images

A dermoscopic lesion area is identified by: Obtaining a dermoscopic image and running a convolutional neural network image classifier on the dermoscopic image to obtain pixelwise lesion prediction scores. Segmenting the dermoscopic image into super-pixels, and computing for each super-pixel an average of the pixelwise prediction scores for pixels within that super-pixel. Computing a mean prediction score across the plurality of super-pixels. Assigning a confidence indicator of '1' to each super-pixel with a prediction score equal or greater than the mean prediction score, and a confidence indicator of '0' to each super-pixel with a prediction score less than the mean prediction score. Constructing a super-pixel graph G=(V,E,W) wherein computing a confidence score function F according to {circumflex over (F)}=arg min(FLF+&#x3bc;&#x2225;F&#x2212;Y&#x2225;); and integrating the confidence score function F with the pixelwise prediction scores to produce a final segmentation of the dermoscopic image into lesion and background areas.

Skin lesion segmentation using deep convolution networks guided by local unsupervised learning

A dermoscopic lesion area is identified by: Obtaining a dermoscopic image and running a convolutional neural network image classifier on the dermoscopic image to obtain pixelwise lesion prediction scores. Segmenting the dermoscopic image into super-pixels, and computing for each super-pixel an average of the pixelwise prediction scores for pixels within that super-pixel. Computing a mean prediction score across the plurality of super-pixels. Assigning a confidence indicator of '1' to each super-pixel with a prediction score equal or greater than the mean prediction score, and a confidence indicator of '0' to each super-pixel with a prediction score less than the mean prediction score. Constructing a super-pixel graph G=(V,E,W) wherein and d=&#x3a3;w; computing a confidence score function F according to {circumflex over (F)}=arg min(FLF+&#x3bc;&#x2225;F&#x2212;Y&#x2225;); and integrating the confidence score function F with the pixelwise prediction scores to produce a final segmentation of the dermoscopic image into lesion and background areas.

A computer-implemented method obtains at least one image from which severity of a given pathological condition presented in the at least one image is to be classified. The method generates a hybrid image representation of the at least one obtained image. The hybrid image representation comprises a concatenation of a discriminative pathology histogram, a generative pathology histogram, and a fully connected representation of a trained baseline convolutional neural network. The hybrid image representation is used to train a classifier to classify the severity of the given pathological condition presented in the at least one image. One non-limiting example of a pathological condition whose severity can be classified with the above method is diabetic retinopathy.

Classification of severity of pathological condition using hybrid image representation

An AMD prediction model utilizes an OCT image estimation model. The OCT image estimation module is created by segmenting an OCT image to generate an OCT projection image for each of multiple biological layers; extracting from each of the generated OCT projection images a first set of features; extracting a second set of features from an input retinal fundus image; for each respective biological layer, registering the input retinal fundus image to the respective OCT projection image by matching at least some of the second set of features with corresponding ones of the first set of features; repeating the above with changes to the input retinal fundus image; and modelling how the changes to the input retinal fundus image are manifest at the correspondingly registered projection images. Estimated OCT projection images can then be generated for the multiple biological layers from a given retinal fundus image.

Early prediction of age related macular degeneration by image reconstruction

Automatically determining image quality of a machine generated image may generate a local saliency map of the image to obtain a set of unsupervised features. The image is run through a trained convolutional neural network (CNN) to extract a set of supervised features from a fully connected layer of the CNN, the image convolved with a set of learned kernels from the CNN to obtain a complementary set of supervised features. The set of unsupervised features and the complementary set of supervised features are combined, and a first decision on gradability of the image is predicted. A second decision on gradability of the image is predicted based on the set of supervised features. Whether the image is gradable is determined based on a weighted combination of the first decision and the second decision.

Growing community of inventors

Kingsville, Australia

Pallab K Roy