Snap Inc.

Microsoft Technology Licensing, LLC

Hao Hu

Kevin Dechau Tang

Kevin Sarabia Dela Rosa

Yanjia Li

Ebony James Charlton

Xing Mei

Venkatraman V Kudallur

Anthony T Chor

Yan Ke

Qiaolin Mao

Ritika Kapadia

Tyler M Barton

Shawn E Pickett

Nomita Paul

James R Wordelman

Aniket A Patankar

Celia Nicole Mourkogiannis

Nathan Kenneth Boyd

Kaveh Anvaripour

Bogdan Maksymchuk

Ekaterina Simbereva

Volodymyr Piven

Annie Huang

Described is a system for performing a set of machine learning model training operations that include: accessing media content items associated with interaction functions initiated by users of an interaction system, generating training data including labels for the media content items, extracting features from a media content item of the media content items, identifying additional media content items to include in the training data based on the extracted features from the media content item, processing the training data using a machine learning model to generate a media content item output; and updating one or more parameters of the machine learning model based on the media content item output. The system checks whether retraining criteria has been met, and repeats the set of machine learning model training operations to retrain the machine learning model.

Machine learning model continuous training system

Method for receiving an input onto a graphical user interface at a client device, capturing an image frame at the client device, the image frame comprising a depiction of an object, identifying the object within the image frame, accessing media content associated with the object within a media repository in response to identifying the object, and causing presentation of the media content within the image frame at the client device.

Contextual media filter search

Systems and methods are provided for performing operations including: receiving, via a messaging application, a request to scan an image captured by a camera of the user device; determining an attribute of an object depicted in the image; identifying, from a plurality of augmented reality items, an augmented reality item that is associated with the attribute of the object; combining the identified augmented reality item with the image captured by the camera; and displaying, via the messaging application, the image captured by the camera combined with the identified augmented reality item.

Augmented reality items based on scan

Aspects of the present disclosure involve a system and a method for performing operations comprising: identifying a plurality of features for frames of a video received by a messaging application server; assigning a first sequence of a first subset of the plurality of features and a second sequence of a second subset of the plurality of features respectively to a first nearest visual codebook cluster and a second nearest visual codebook cluster; applying the first and second nearest visual codebook clusters to a visual search database to identify a plurality of candidate matching videos; selecting a given matching video from the plurality of candidate matching videos based on a rank representing similarity of the given matching video to the video received by the messaging application server; and accessing an augmented reality experience corresponding to the given matching video.

Video matching with a messaging application

Aspects of the present disclosure involve a system and a method for performing operations comprising: identifying a plurality of features for an image received by a messaging application server; assigning a first of the plurality of features and a second of the plurality of features respectively to a first nearest visual codebook cluster and a second nearest visual codebook cluster; applying the first and second nearest visual codebook clusters to a visual search database to identify a plurality of candidate matching images; selecting a given matching image based on a geometric verification of the plurality of matching images and the received image; and accessing an augmented reality experience corresponding to the given matching image.

Visual matching with a messaging application

Systems and methods are provided for performing operations including receiving, via a messaging application, a request to scan an image captured by a camera of the user device; determining an attribute of an object depicted in the image; identifying, from a plurality of augmented reality items, an augmented reality item that is associated with the attribute of the object; combining the identified augmented reality item with the image captured by the camera; and displaying, via the messaging application, the image captured by the camera combined with the identified augmented reality item.

Aspects of the present disclosure involve a system and a method for performing operations comprising: capturing, by a client device implementing a messaging application, an image; receiving, from a server, an identification of an augmented reality experience associated with the image, the server identifying the augmented reality experience by assigning visual words to features of the image and searching a visual search database to identify a plurality of marker images, and the server retrieving the augmented reality experience associated with a given one of the plurality of marker images; automatically, in response to capturing the image, displaying one or more graphical elements of the identified augmented reality experience; and receiving input from the user interacting with the one or more graphical elements.

3D graphic interaction based on scan

Aspects of the present disclosure involve a system and a method for performing operations comprising: capturing an image that depicts currency, receiving an identification of an augmented reality experience associated with the image, the server identifying the augmented reality experience by assigning visual words to features of the image and searching a visual search database to identify a plurality of marker images associated with one or more charities, and the server retrieving the augmented reality experience associated with a given one of the plurality of marker images; automatically, in response to capturing the image that depicts currency, displaying one or more graphical elements of the identified augmented reality experience that represent a charity of the one or more charities; and receiving a donation to the charity from a user of the client device in response to an interaction with the one or more graphical elements that represent the charity.

Online transaction based on currency scan

Content pre-render and pre-fetch techniques are described. In one or more implementations, a likelihood is predicted of subsequent navigation to content associated with a network address based at least in part on monitored navigation performed by a computing device to one or more network addresses. A determination is then made whether to cause the computing device to pre-fetch and pre-render the content based at least in part on the prediction.

Growing community of inventors

Bellevue, WA, United States of America

Hao Hu