Method and system for image categorization using a visual language model

Abstract

Methods, systems, and techniques for image categorization using a visual language model. A set of images is clustered into clusters respectively corresponding to differently categorized objects. Names are respectively assigned to the clusters, and image captions are respectively generated for the clusters using the names. The image captions and respective images represent image-text pairs. Those image-text pairs are input to the visual language model as context for a query. The query is then input to the visual language model. The query includes a request to categorize a query image of a class represented in the context. In response to the query, the visual language model performs an open-ended generative categorization of the query image.