Naver Corporation

Naver Labs Corporation

Cesar De Souza

Rafael Sampaio De Rezende

Naila Murray

Jon Almazan

Diane Larlus

Jérome Revaud

Philippe Weinzaepfel

Martin Humenberger

A system includes: a training dataset including first objects of a first modality and second objects of a second modality different than the first modality, where the second objects include text that is descriptive of the first objects; a first matrix including first relevance values indicative of relevance between the first objects and the second objects, respectively; a second matrix including second relevance values indicative of relevance between the second objects and the first objects, respectively; and a training module configured to: assign ones of the second objects to bins based on distances between the ones of the objects and a query; determine a ranking measure based on a number of the ones of the second objects assigned to the bins; determine losses based on the ranking measure and the first and second matrices; determine a final loss based on the losses; train embedding functions based on the final loss.

Cross-modal search systems and methods

A training system includes: a training dataset including first objects of a first modality and second objects of a second modality that are associated with the first objects, respectively; a first matrix including first relevance values indicative of relevance between the first objects and the second objects, respectively; a second matrix including second relevance values indicative of relevance between the second objects and the first objects, respectively; and a training module configured to: based on similarities between ones of the second objects, generate a third matrix by selectively adding first additional relevance values to the first matrix; based on the similarities between the ones of the second objects, generate a fourth matrix by selectively adding second additional relevance values to the second matrix; and store the third and fourth matrices in memory of a search module for cross-modal retrieval in response to receipt of search queries.

A method of performing image retrieval includes: obtaining a query image; generating a global feature descriptor of the query image by inputting the query image into a convolutional neural network (CNN) and obtaining the global feature descriptor as an output of the CNN, where parameters of the CNN are learned during training of the CNN on a batch of training images using a listwise ranking loss function and optimizing a quantized mean average precision ranking evaluation metric; determining similarities between the query image and other images based on distances between the global feature descriptor of the query image and global feature descriptors of the other images, respectively; ranking the other images based on the similarities, respectively; and selecting a set of the other images based on the similarities between the query image and the other images.

Training a convolutional neural network for image retrieval with a listwise ranking loss function

A system for detecting and describing keypoints in images is described. A camera is configured to capture an image including a plurality of pixels. A fully convolutional network is configured to jointly and concurrently: generate descriptors for each of the pixels, respectively; generate reliability scores for each of the pixels, respectively; and generate repeatability scores for each of the pixels, respectively. A scoring module is configured to generate scores for the pixels, respectively, based on the reliability scores and the repeatability scores of the pixels, respectively. A keypoint list module is configured to: select X of the pixels having the X highest scores, where X is an integer greater than 1; and generate a keypoint list including: locations of the selected X pixels; and the descriptors of the selected X pixels.

Growing community of inventors

Grenoble, France

Cesar De Souza