Snap Inc.

Frank Lu

Riza Alp Guler

Haoyang Wang

Itamar Berger

Matan Zohar

Avihay Assouline

Georgios Papandreou

Antonios Kakolyris

Methods and systems are disclosed for generating a body mesh from a single image. The system predicts both a volumetric reconstruction tensor of the monocular image and a pose of an object by applying a first machine learning model to a monocular image. The system identifies a portion of the pose of the object that corresponds to a point in a canonical space associated with a set of position encoding information. The system obtains a point of the volumetric reconstruction tensor corresponding to the identified portion of the pose. The system classifies the obtained point as being inside or outside of a canonical volume by applying a second machine learning model to the obtained point of the volumetric reconstruction tensor together with the set of position encoding information. The system generates a three-dimensional (3D) mesh representing the object in the canonical space.

Body mesh reconstruction from RGB image

Aspects of the present disclosure involve a system for providing virtual experiences. The system accesses, by a messaging application, an image depicting a person. The system generates, by the messaging application, a three-dimensional (3D) avatar based on the person depicted in the image. The system receives input that selects a pose for the 3D avatar and one or more fashion items to be worn by the 3D avatar and places, by the messaging application, the 3D avatar in the selected pose and wearing the one or more fashion items in an augmented reality (AR) experience.

Applying animated 3D avatar in AR experiences

Growing community of inventors

London, United Kingdom

Frank Lu