Naver Corporation

Naver Labs Corporation

Philippe Weinzaepfel

Boris Chidlovskii

Gregory Rogez

Romain Bregier

Ioannis Kalantidis

Diane Larlus

Vincent Leroy

Hadrien Combaluzier

Martin Humenberger

Gabriela Csurka

Cordelia Luise Schmid

Jérome Revaud

Cesar De Souza

Thibault Groueix

Jérôme Revaud

Yohann Cabon

Mert Bulent Sariyildiz

Vasileios Choutas

Fabien Baradel

Thomas Lucas

Noé Pion

A training system includes: a neural network model configured to determine three-dimensional coordinates of joints, respectively, representing poses of animals in images, where the neural network model is trained using a first training dataset including: images including animals; and coordinates of joints of the animals in the images, respectively; and a training module configured to, after the training of the neural network model using the first training dataset, train the neural network model using a second training dataset including motion capture data, where the motion capture data does not include images of animals and includes measured coordinates at points, respectively, on animals.

Pose estimation systems and methods trained using motion capture data

A computer-implemented method of recognition of actions performed by individuals includes: by one or more processors, obtaining images including at least a portion of an individual; by the one or more processors, based on the images, generating implicit representations of poses of the individual in the images; and by the one or more processors, determining an action performed by the individual and captured in the images by classifying the implicit representations of the poses of the individual.

Action recognition using implicit pose representations

Am image retrieval system includes: a neural network (NN) module configured to generate local features based on an input image; an iterative attention module configured to, via T iterations, generate an ordered set of super features in the input image based on the local features, where T is an integer greater than 1; and a selection module configured to select a second image from a plurality of images in an image database based on the second image having a second ordered set of super features that most closely match the ordered set of super features in the input image, where the super features in the set of super features do not include redundant local features of the input image.

Systems and methods for image retrieval using super features

A training system includes: an encoder module configured to receive a query image and to generate a first vector representative of one or more features in the query image using an encoder; a mixing module configured to generate a second vector by mixing a third vector, representative of one or more features in a second image that is classified as a negative relative to the query image, with a fourth vector; and an adjustment module configured to train the encoder by selectively adjusting one or more parameters of the encoder based on the first vector and the second vector.

Unsupervised contrastive learning of visual representations using negative mixing

A system for generating whole body poses includes: a body regression module configured to generate a first pose of a body of an animal in an input image by regressing from a stored body anchor pose; a face regression module configured to generate a second pose of a face of the animal in the input image by regressing from a stored face anchor pose; an extremity regression module configured to generate a third pose of an extremity of the animal in the input image by regressing from a stored extremity anchor pose; and a pose module configured to generate a whole body pose of the animal in the input image based on the first pose, the second pose, and the third pose.

Distillation of part experts for whole-body pose estimation

A method for training a convolutional recurrent neural network for semantic segmentation in videos, includes (a) training, using a set of semantically segmented training images, a first convolutional neural network; (b) training, using a set of semantically segmented training videos, a convolutional recurrent neural network, corresponding to the first convolutional neural network, wherein a convolutional layer has been replaced by a recurrent module having a hidden state. The training of the convolutional recurrent neural network, for each pair of successive frames (t&#x2212;1, t&#x2208;1; T) of a video of the set of semantically segmented training videos includes warping an internal state of a recurrent layer according to an estimated optical flow between the frames of the pair of successive frames, so as to adapt the internal state to the motion of pixels between the frames of the pair and learning parameters of at least the recurrent module.

Method for training a convolutional recurrent neural network and for semantic segmentation of inputted video using the trained convolutional recurrent neural network

A system for detecting and describing keypoints in images is described. A camera is configured to capture an image including a plurality of pixels. A fully convolutional network is configured to jointly and concurrently: generate descriptors for each of the pixels, respectively; generate reliability scores for each of the pixels, respectively; and generate repeatability scores for each of the pixels, respectively. A scoring module is configured to generate scores for the pixels, respectively, based on the reliability scores and the repeatability scores of the pixels, respectively. A keypoint list module is configured to: select X of the pixels having the X highest scores, where X is an integer greater than 1; and generate a keypoint list including: locations of the selected X pixels; and the descriptors of the selected X pixels.

Joint detection and description systems and methods

A method for training, using a plurality of training images with corresponding six degrees of freedom camera pose for a given environment and a plurality of reference images, each reference image depicting an object-of-interest in the given environment and having a corresponding two-dimensional to three-dimensional correspondence for the given environment, a neural network to provide visual localization by: for each training image, detecting and segmenting object-of-interest in the training image; generating a set of two-dimensional to two-dimensional matches between the detected and segmented objects-of-interest and corresponding reference images; generating a set of two-dimensional to three-dimensional matches from the generated set of two-dimensional to two-dimensional matches and the two-dimensional to three-dimensional correspondences corresponding to the reference images; and determining localization, for each training image, by solving a perspective-n-point problem using the generated set of two-dimensional to three-dimensional matches.

System and method for training a neural network for visual localization based upon learning objects-of-interest dense match regression

A method for training a convolutional neural network for classification of actions performed by subjects in a video is realized by (a) for each frame of the video, for each key point of the subject, generating a heat map of the key point representing a position estimation of the key point within the frame; (b) colorizing each heat map as a function of the relative time of the corresponding frame in the video; (c) for each key point, aggregating all the colorized heat maps of the key point into at least one image representing the evolution of the position estimation of the key point during the video; and training the convolutional neural network using as input the sets associated to each training video of images representing the evolution of the position estimation of each key point during the video.

System and method for training a convolutional neural network and classifying an action performed by a subject in a video using the trained convolutional neural network

Growing community of inventors

Montbonnot-Saint-Martin, France

Philippe Weinzaepfel