Google Inc.

Deqing Sun

Ming-Hsuan Yang

Sofien Bouaziz

Ruofei Du

Sean Ryan Francesco Fanello

Rohit Kumar Pandey

Cem Keskin

Yinda Zhang

Danhang Tang

Mingsong Dou

Qifei Wang

Taihong Xiao

Jinwei Yuan

Feitong Tan

Kaiwen Guo

Systems and methods for training models to predict dense correspondences across images such as human images. A model may be trained using synthetic training data created from one or more 3D computer models of a subject. In addition, one or more geodesic distances derived from the surfaces of one or more of the 3D models may be used to generate one or more loss values, which may in turn be used in modifying the model's parameters during training.

Systems and methods for training models to predict dense correspondences in images using geodesic distances

A method includes obtaining a first plurality of feature vectors associated with a first image and a second plurality of feature vectors associated with a second image. The method also includes generating a plurality of transformed feature vectors by transforming each respective feature vector of the first plurality of feature vectors by a kernel matrix trained to define an elliptical inner product space. The method additionally includes generating a cost volume by determining, for each respective transformed feature vector of the plurality of transformed feature vectors, a plurality of inner products, wherein each respective inner product of the plurality of inner products is between the respective transformed feature vector and a corresponding candidate feature vector of a corresponding subset of the second plurality of feature vectors. The method further includes determining, based on the cost volume, a pixel correspondence between the first image and the second image.

Learnable cost volume for determining pixel correspondence

Growing community of inventors

Cambridge, MA, United States of America

Deqing Sun