Snap Inc.

Kevin Sarabia Dela Rosa

Hao Hu

Bogdan Maksymchuk

Ekaterina Simbereva

Volodymyr Piven

Yanjia Li

Patrick Poirson

Kwot Sin Lee

Jin Cheng

Annie Huang

Adel Elmalaha

Described is a system for performing a set of machine learning model training operations that include: accessing media content items associated with interaction functions initiated by users of an interaction system, generating training data including labels for the media content items, extracting features from a media content item of the media content items, identifying additional media content items to include in the training data based on the extracted features from the media content item, processing the training data using a machine learning model to generate a media content item output; and updating one or more parameters of the machine learning model based on the media content item output. The system checks whether retraining criteria has been met, and repeats the set of machine learning model training operations to retrain the machine learning model.

Machine learning model continuous training system

An image based valuation system to perform operations that include: causing display of image data at a client device, the image data comprising a depiction of an object that comprises a set of object features; detecting the object based on the depiction of the object within the image data, the object corresponding with an object class; identifying the set of object features that correspond with the object; accessing a repository that corresponds with the object class, the repository comprising at least an indication of a valuation of the object based on the set of object features; and causing display of a presentation of the valuation at the client device.

Image based valuation system

Embodiments herein describe a portal system for maintaining a user portal that is used for managing and generating scannable images. The scannable images may be linked to different types of content such as website URLs, social media platform user profiles, and augmented reality experiences. The user portal is accessible from a social media platform via user login credentials. Upon entering their user login credentials, the portal system authenticates the user login credentials and displays a list existing scannable images that are stored in association with the user portal. The user may modify one or more aspects of the existing scannable images or may generate a new scannable image from the user interface.

Unlockable content creation portal

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: receiving an image from a client device; applying a machine learning model to the image to generate an embedding query vector, the machine learning model being trained to encode a plurality of images and text into a common embedding space; searching, based on the embedding query vector, a database of augmented reality (AR) experiences to identify a subset of AR experiences associated with one or more embeddings that correspond to the embedding query vector; and transmitting to the client device the subset of AR experiences associated with the one or more embeddings that correspond to the embedding query vector.

Searching augmented reality experiences using visual embeddings

Aspects of the present disclosure involve a system and a method for performing operations comprising: identifying a plurality of features for frames of a video received by a messaging application server; assigning a first sequence of a first subset of the plurality of features and a second sequence of a second subset of the plurality of features respectively to a first nearest visual codebook cluster and a second nearest visual codebook cluster; applying the first and second nearest visual codebook clusters to a visual search database to identify a plurality of candidate matching videos; selecting a given matching video from the plurality of candidate matching videos based on a rank representing similarity of the given matching video to the video received by the messaging application server; and accessing an augmented reality experience corresponding to the given matching video.

Video matching with a messaging application

Aspects of the present disclosure involve a system and a method for performing operations comprising: identifying a plurality of features for an image received by a messaging application server; assigning a first of the plurality of features and a second of the plurality of features respectively to a first nearest visual codebook cluster and a second nearest visual codebook cluster; applying the first and second nearest visual codebook clusters to a visual search database to identify a plurality of candidate matching images; selecting a given matching image based on a geometric verification of the plurality of matching images and the received image; and accessing an augmented reality experience corresponding to the given matching image.

Visual matching with a messaging application

Aspects of the present disclosure involve a system and a method for performing operations comprising: capturing, by a client device implementing a messaging application, an image; receiving, from a server, an identification of an augmented reality experience associated with the image, the server identifying the augmented reality experience by assigning visual words to features of the image and searching a visual search database to identify a plurality of marker images, and the server retrieving the augmented reality experience associated with a given one of the plurality of marker images; automatically, in response to capturing the image, displaying one or more graphical elements of the identified augmented reality experience; and receiving input from the user interacting with the one or more graphical elements.

3D graphic interaction based on scan

Aspects of the present disclosure involve a system and a method for performing operations comprising: capturing an image that depicts currency, receiving an identification of an augmented reality experience associated with the image, the server identifying the augmented reality experience by assigning visual words to features of the image and searching a visual search database to identify a plurality of marker images associated with one or more charities, and the server retrieving the augmented reality experience associated with a given one of the plurality of marker images; automatically, in response to capturing the image that depicts currency, displaying one or more graphical elements of the identified augmented reality experience that represent a charity of the one or more charities; and receiving a donation to the charity from a user of the client device in response to an interaction with the one or more graphical elements that represent the charity.

Growing community of inventors

Seattle, WA, United States of America

Kevin Sarabia Dela Rosa