Snap Inc.

Jean Luo

Celia Nicole Mourkogiannis

Alek Matthiessen

Virginia Drummond

Vincent Sung

Christine Barron

Jonathan Solichin

Olesia Voronova

Oleksandr Grytsiuk

Ibram Uppal

Kaveh Anvaripour

Ilteris Kaan Canberk

Samir Ahmed

Riccardo Boscolo

Yang Wen

Amit Gaur

Ashley Michelle Wayne

Ivan Golub

Bogdan Maksymchuk

Erika Michele Kehrwald

Oleksandr Chepizhenko

Itamar Berger

Matan Zohar

Avihay Assouline

Grygoriy Kozhemiak

Trevor Stephenson

Carolina Arguelles

Chenguang Liu

Oleksii Gordiienko

Per Sandell

Mykhailo Sokolov

Yichuan Wang

Maxwell Gale

Charles Melbye

Shannon Ward

Ruoyu Li

Zhehao Zhou

Yuliia Khoryshko

Evelina Yatselenko

Serhii Dehtiarov

Stiven Deleur

Alona Kotliar

The subject technology receives metadata corresponding to a respective media overlay, the metadata including information indicating that the respective media overlay is configured to be applied as an image processing operation during post-processing of image data during a post-capture stage. The subject technology selects the respective media overlay in response to the information indicating that the respective media overlay is configured to be applied as an image processing operation during post-processing of image data. The subject technology, based at least in part on a category indicator associated with the respective media overlay, populates a group of media overlays with at least the respective media overlay. The subject technology sends, to a client electronic device, second metadata including at least information related to the group of media overlays.

Providing post-capture media overlays for post-capture processing in a messaging system

Systems, methods, and computer readable media for augmented reality beauty product tutorials. Methods disclose determining from live images of an augmented reality (AR) tutorial effects, the effects indicating changes to the live images of a presenter of the AR tutorial from a beauty product being applied to a body part of the presenter. The methods further comprising determining from the live images motion, the motion indicating motion of the beauty product from the beauty product being applied to the body part of the presenter and storing the effects and the motion.

Augmented reality beauty product tutorials

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing make-up based augmented reality content. The program and method provide for receiving a request to present augmented reality content in association with a captured image depicting a face of the user; accessing an augmented reality content item associated with applying makeup to the face and configured to generate a mesh for tracking plural regions of the face; receiving user input selecting a region; determining at least one of a range of color values or a range of contrast values relating to available makeup products for the selected region; and presenting an interface element in association with the face, the interface element for user selection of at least one of a color value within the range of color values or a contrast value within the range of contrast values.

Selecting color values for augmented reality-based makeup

The subject technology receives image data including a representation of a physical item. The subject technology analyzes the image data to determine an object corresponding to the physical item. The subject technology identifies a set of colors corresponding to a set of regions of the determined object. The subject technology analyzes second image data to detect a second object corresponding to a representation of a particular body part of a user. The subject technology generates augmented reality content based at least in part on the identified set of colors and the detected second object. The subject technology causes display, at a client device, the augmented reality content applied to the detected second object.

Augmented reality experiences of color palettes in a messaging system

The subject technology receives image data including a representation of a physical item. The subject technology analyzes the image data to determine an object corresponding to the physical item. The subject technology extracts product metadata based on the determined object. The subject technology sends, to a server, the product metadata to determine second product metadata associated with the product metadata. The subject technology receives, from the server, the second product metadata, the second product metadata including additional information related to the physical item. The subject technology causes display, at a client device, the additional information related to the physical item based at least in part on the second product metadata.

Identification of physical products for augmented reality experiences in a messaging system

In one or more implementations, user interfaces may be generated that include user interface elements that enable the tracking or monitoring of interactions with augmented reality content by users of a client application. Usage metrics for augmented reality content may be collected and analyzed. In addition, information about client application users that interact with augmented reality content may be generated. In one or more examples, characteristics of users of the client application that interact with one or more augmented reality content items may be analyzed to determine a user characterization for one or more augmented reality content items. The usage metrics and user characterizations for augmented reality content items may be presented via a dashboard that is accessible to creators of augmented reality content.

Analyzing augmented reality content usage data

In one or more implementations, information about a number of products may be obtained. Visual effects corresponding to each product may be applied to objects included in user content to change the appearance of the objects included in the user content. Augmented reality content may be implemented to cause changes to the appearance of one or more objects included in the user content. In various examples, a number of visual effects related to different products may be applied to objects included in the user content. A user interface including information about each of the products applied to objects included in the user content may be produced.

Generating augmented reality content based on user-selected product data

Methods and systems are disclosed for performing operations comprising: receiving an image that includes a depiction of a face of a user; generating a plurality of landmarks of the face based on the received image; removing a set of interfering landmarks from the plurality of landmarks resulting in a remaining set of landmarks of the plurality of landmarks; obtaining a depth map for the face of the user; and computing a real-world scale of the face of the user based on the depth map and the remaining set of landmarks.

True size eyewear in real time

The present invention relates to improvements to systems and methods for determining a current location of a client device, and for identifying and selecting appropriate geo-fences based on the current location of the client device. An improved geo-fence selection system performs operations that include associating media content with a geo-fence that encompasses a portion of a geographic region, sampling location data from a client device, defining a boundary based on the sampled location data from the client device, detecting an overlap between the boundary and the geo-fence, retrieving the media content associated with the geo-fence, and loading the media content at a memory location of the client device, in response to detecting the overlap.

Geo-fence selection system

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing augmented reality-based makeup. The program and method provide for receiving a request to present augmented reality content in association with an image captured by a device camera, the image depicting a user's face; accessing an augmented reality content item applying makeup to the face, the augmented reality content configured to generate a mesh for tracking plural regions of the face and to present available makeup products with respect to the plural regions; presenting the augmented reality content item in association with the face depicted in the image; receiving user input selecting a region of the plural regions; determining a set of available makeup products corresponding to the selected region; and updating presentation of the augmented reality content item based on the set of available makeup products.

Providing augmented reality-based makeup in a messaging system

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing augmented reality-based makeup. The program and method provide for receiving, by a messaging application running on a device of a user, a request to present augmented reality content in association with an image captured by a device camera, the image depicting a face of the user; accessing an augmented reality content item configured to generate a plurality of completed looks with respect to applying makeup to the face; presenting the augmented reality content item, including the plurality of completed looks, in association with the face depicted in the image; receiving user input selecting a completed look of the plurality of completed looks; and displaying, in response to receiving the user input, an interface with a set of makeup products associated with the selected completed look.

Method, device, and non-transitory computer-readable storage medium for providing augmented reality-based makeup product sets in a messaging system

Systems, methods, and computer-readable media for adding beauty products to tutorials are presented. Methods include accessing video data comprising images of a presenter creating a tutorial, the tutorial depicting the presenter applying a beauty product to a body part of the presenter. Methods further include processing the video data to identify changes to the body part of the presenter from an application of the beauty product, and responding to identifying changes to the body part of the presenter from the application of the beauty product by processing the video data to identify the beauty product. Methods further include retrieving information regarding the beauty product and causing presentation of information regarding the beauty product on a display device.

Adding beauty products to augmented reality tutorials

Systems and methods described herein provide for retrieving, from a storage device, first image data previously captured by a client device. The systems and methods further detect a selection of a first image processing operation and perform the first image processing operation on the first image data to generate second image data. The systems and methods further detect a selection of a second image processing operation and perform the second image processing operation on the second image data to generate third image data. The systems and methods generate a message comprising the third image data.

Layering of post-capture processing in a messaging system

Usage metrics for augmented reality content may be identified and analyzed to determine measures of fitness for respective usage metrics. The measures of fitness may indicate a level of correlation with an outcome specified by an augmented reality content creator and an amount of interaction with an augmented reality content item by users of a client application. Recommendations may be provided to augmented reality content creators indicating modifications to augmented reality content items that have at least a threshold probability of increasing the level of interaction between users of the client application and the augmented reality content item.

Analyzing augmented reality content item usage data

The subject technology selects a set of augmented reality content generators from a plurality of augmented reality content generators. The subject technology causes display, at a client device, of a graphical interface comprising a plurality of selectable graphical items. The subject technology receives, at the client device, a selection of a first selectable graphical item from the plurality of selectable graphical items, the first selectable graphical item comprising a first augmented reality content generator corresponding to a particular geolocation. The subject technology causes display, at the client device, at least one augmented content reality item generated by the first augmented reality content generator. The subject technology causes display, at a second client device, the at least one augmented content reality item generated by the first augmented reality content generator, the at least one augmented content reality item comprising the 3D scene based at least in part on the particular geolocation.

Growing community of inventors

Los Angeles, CA, United States of America

Jean Luo