Niantic, Inc.

Niantic Spatial, Inc.

Rigel Gareth Benton

Erik Murphy-Chutorian

Dat Chu

Scott Pollack

Nicholas Butko

Nathan Waters

Alvin Portillo

Joshua Aisheng Mao

The present disclosure describes an online system that applies template-based facial augmentations to images as part of an augmented reality experience. A facial augmentation template is a template for an augmented reality (AR) modification to apply to a user's face. These templates include a structure for how the augmentation should be applied to the face and display parameters that a user can set for how the templates should be rendered (e.g., color, specularity, or opacity). The online system identifies feature points on a 3D model of the user's face that correspond to features on the user's actual face, and localizes the template relative to the 3D model by mapping anchor points of the template to corresponding feature points on the 3D model. The online system renders the facial augmentation template at the localization to generate an augmented image of the user's face.

Template-based AR facial augmentation

Implementations generally relate to providing augmented reality in a web browser. In one implementation, a method includes capturing images of a physical scene with a camera of a device. The method further includes determining motion of the camera using six degrees of freedom (6DoF) markerless tracking. The method further includes overlaying virtual three-dimensional (3D) content onto a depicted physical scene in the images, resulting in augmented reality (AR) images. The method further includes rendering the AR images in a browser of the device.

Efficient GPU/CPU pipeline for providing augmented reality content

Growing community of inventors

Mountain View, CA, United States of America

Rigel Gareth Benton