Flexible framework for joint representation learning and unknown category discovery

Abstract

Systems and methods for providing deep learning models capable of performing joint representation learning and new category discovery on a mixture of labeled and unlabeled data, which may include single- and multi-modal data. In some examples, a flexible end-to-end framework uses unified contrastive learning on labeled and unlabeled data based on both instance discrimination and category discrimination, and further uses Winner-Take-All hashing to generate a pseudo-label based on the similarity between each pair of unlabeled data points that can be used to train the model to generate clustering assignments for each unlabeled data point. In some examples, the unified contrastive learning may be further based on cross-modal discrimination.