Systems, methods, and apparatuses for implementing self-supervised visual representation learning using order and appearance recovery on a vision transformer

Abstract

Described herein are means for performing self-supervised visual representation learning using order and appearance recovery on a vision transformer. An exemplary system having a processor and memory is specially configured to execute instructions including: receiving medical image training data; selecting a medical image; generating a first perturbed image by applying local pixel shuffling and other image perturbations and outputting a first patchified perturbed image; generating a second randomized patchified image by patchifying and applying a random permutation to the original image; inputting the first patchified perturbed image and the second randomized patchified image into first and second transformer encoders which each generate and then share first and second generated weights through the recovery of both and patch order appearance from each image; and outputting a pre-trained AI model to perform medical image diagnosis on a new medical image absent from the training data input received by the system.