Cross-modal search systems and methods

Abstract

A training system includes: a training dataset including first objects of a first modality and second objects of a second modality that are associated with the first objects, respectively; a first matrix including first relevance values indicative of relevance between the first objects and the second objects, respectively; a second matrix including second relevance values indicative of relevance between the second objects and the first objects, respectively; and a training module configured to: based on similarities between ones of the second objects, generate a third matrix by selectively adding first additional relevance values to the first matrix; based on the similarities between the ones of the second objects, generate a fourth matrix by selectively adding second additional relevance values to the second matrix; and store the third and fourth matrices in memory of a search module for cross-modal retrieval in response to receipt of search queries.