Arizona State University

Fatemeh Haghighi

Jianming Liang

Mohammad Reza Hosseinzadeh Taher

Zongwei Zhou

Jiaxuan Pang

Nahid Ul Islam

DongAo Ma

A self-supervised learning framework for empowering instance discrimination in medical imaging using Context-Aware instance Discrimination (CAiD), in which the trained deep models are then utilized for the processing of medical imaging. An exemplary system receives a plurality of medical images; trains a self-supervised learning framework to increasing instance discrimination for medical imaging using a Context-Aware instance Discrimination (CAiD) model using the received plurality of medical images; generates multiple cropped image samples and augments samples using image distortion; applies instance discrimination learning a mapping back to a corresponding original image; reconstructs the cropped image samples and applies an auxiliary context-aware learning loss operation; and generates as output, a pre-trained CAiD model based on the application of both (i) the instance discrimination learning and (ii) the auxiliary context-aware learning loss operation.

Systems, methods, and apparatuses for implementing a self-supervised learning framework for empowering instance discrimination in medical imaging using context-aware instance discrimination (CAiD)

Described herein are means for the generation of Transferable Visual Word (TransVW) models through self-supervised learning in the absence of manual labeling, in which the trained TransVW models are then utilized for the processing of medical imaging. For instance, an exemplary system is specially configured to perform self-supervised learning for an AI model in the absence of manually labeled input, by performing the following operations: receiving medical images as input; performing a self-discovery operation of anatomical patterns by building a set of the anatomical patterns from the medical images received at the system, performing a self-classification operation of the anatomical patterns; performing a self-restoration operation of the anatomical patterns within cropped and transformed 2D patches or 3D cubes derived from the medical images received at the system by recovering original anatomical patterns to learn different sets of visual representation; and providing a semantics-enriched pre-trained AI model having a trained encoder-decoder structure with skip connections in between based on the performance of the self-discovery operation, the self-classification operation, and the self-restoration operation. Other related embodiments are disclosed.

Systems, methods, and apparatuses for implementing transferable visual words by exploiting the semantics of anatomical patterns for self-supervised learning

Described herein are systems, methods, and apparatuses for implementing self-supervised domain-adaptive pre-training via a transformer for use with medical image classification in the context of medical image analysis. An exemplary system includes means for receiving a first set of training data having non-medical photographic images; receiving a second set of training data with medical images; pre-training an AI model on the first set of training data with the non-medical photographic images; performing domain-adaptive pre-training of the AI model via self-supervised learning operations using the second set of training data having the medical images; generating a trained domain-adapted AI model by fine-tuning the AI model against the targeted medical diagnosis task using the second set of training data having the medical images; outputting the trained domain-adapted AI model; and executing the trained domain-adapted AI model to generate a predicted medical diagnosis from an input image not present within the training data.

Systems, methods, and apparatuses for implementing self-supervised domain-adaptive pre-training via a transformer for use with medical image classification

A Discriminative, Restorative, and Adversarial (DiRA) learning framework for self-supervised medical image analysis is described. For instance, a pre-trained DiRA framework may be applied to diagnosis and detection of new medical images which form no part of the training data. The exemplary DiRA framework includes means for receiving training data having medical images therein and applying discriminative learning, restorative learning, and adversarial learning via the DiRA framework by cropping patches from the medical images; inputting the cropped patches to the discriminative and restorative learning branches to generate discriminative latent features and synthesized images from each; and applying adversarial learning by executing an adversarial discriminator to perform a min-max function for distinguishing the synthesized restorative image from real medical images. The pre-trained model of the DiRA framework is then provided as output for use in generating predictions of disease within medical images.

Systems, methods, and apparatuses for implementing discriminative, restorative, and adversarial (DiRA) learning for self-supervised medical image analysis

Described herein are means for the generation of semantic genesis models through self-supervised learning in the absence of manual labeling, in which the trained semantic genesis models are then utilized for the processing of medical imaging. For instance, an exemplary system is specially configured with means for performing a self-discovery operation which crops 2D patches or crops 3D cubes from similar patient scans received at the system as input; means for transforming each anatomical pattern represented within the cropped 2D patches or the cropped 3D cubes to generate transformed 2D anatomical patterns or transformed 3D anatomical patterns; means for performing a self-classification operation of the transformed anatomical patterns by formulating a C-way multi-class classification task for representation learning; means for performing a self-restoration operation by recovering original anatomical patterns from the transformed 2D patches or transformed 3D cubes having transformed anatomical patterns embedded therein to learn different sets of visual representation; and means for providing a semantics-enriched pre-trained AI model having a trained encoder-decoder structure with skip connections in between based on the performance of the self-discovery operation, the self-classification operation, and the self-restoration operation. Other related embodiments are disclosed.

Systems, methods, and apparatuses for the use of transferable visual words for AI models through self-supervised learning in the absence of manual labeling for the processing of medical imaging

Described herein are means for learning semantics-enriched representations via self-discovery, self-classification, and self-restoration in the context of medical imaging. Embodiments include the training of deep models to learn semantically enriched visual representation by self-discovery, self-classification, and self-restoration of the anatomy underneath medical images, resulting in a collection of semantics-enriched pre-trained models, called Semantic Genesis. Other related embodiments are disclosed.

Systems, methods, and apparatuses for learning semantics-enriched representations via self-discovery, self-classification, and self-restoration in the context of medical imaging

Not only is annotating medical images tedious and time consuming, but it also demands costly, specialty-oriented expertise, which is not easily accessible. To address this challenge, a new self-supervised framework is introduced: TransVW (transferable visual words), exploiting the prowess of transfer learning with convolutional neural networks and the unsupervised nature of visual word extraction with bags of visual words, resulting in an annotation-efficient solution to medical image analysis. TransVW was evaluated using NIH ChestX-ray14 to demonstrate its annotation efficiency. When compared with training from scratch and ImageNet-based transfer learning, TransVW reduces the annotation efforts by 75% and 12%, respectively, in addition to significantly accelerating the convergence speed. More importantly, TransVW sets new records: achieving the best average AUC on all 14 diseases, the best individual AUC scores on 10 diseases, and the second best individual AUC scores on 3 diseases. This performance is unprecedented, because heretofore no self-supervised learning method has outperformed ImageNet-based transfer learning and no annotation reduction has been reported for self-supervised learning. These achievements are contributable to a simple yet powerful observation: The complex and recurring anatomical structures in medical images are natural visual words, which can be automatically extracted, serving as strong yet free supervision signals for CNNs to learn generalizable and transferable image representation via self-supervision.

Systems, methods, and apparatuses for implementing a self-supervised chest x-ray image analysis machine-learning model utilizing transferable visual words

Growing community of inventors

Tempe, AZ, United States of America

Fatemeh Haghighi