Self-supervised learning for interventional image analysis

Abstract

Systems and methods for performing one or more medical imaging analysis tasks are provided. A sequence of medical images is received. One or more patches are extracted from each image of the sequence of medical images. Spatio-temporal features are extracted from the one or more extracted patches using a machine learning based encoder network. One or more medical imaging analysis tasks are performed based on the extracted spatio-temporal features. Results of the one or more medical imaging analysis tasks are output. The machine learning based encoder network is trained by receiving a sequence of training medical images. Patches of a first set of images of the sequence of training medical images are masked according to a first masking strategy. Patches of a second set of images of the sequence of training medical images are masked according to a second masking strategy. The machine learning based encoder network is trained to learn a spatio-temporal relationship between the unmasked patches of the first set of images and the second set of images. The machine learning based encoder network is output.