Snap Inc.

Daniel Moreno

Kyle Goodrich

Andrew James McPhee

Maxim Maximov Lazarov

Samuel Edward Hare

Tony Mathew

Wentao Shang

Dhritiman Sagar

Dylan Shane Eirinberg

Kaveh Anvaripour

Christie Marie Heikkinen

Nathan Kenneth Boyd

Ilteris Kaan Canberk

David Meisenholder

Pedram Javidpour

Jeremy Baker Voss

Celia Nicole Mourkogiannis

Karen Stolzenberg

Christine Barron

Mathieu Emmanuel Vignau

Laurent Desserrey

Piers George Cowburn

Ebony James Charlton

David Phillip Taitz

David Li

Shin Hwun Kang

Joseph Timothy Fortier

Sergey Tulyakov

Patrick Mandia

Isac Andreas Müller Sandvik

Alex Edelsburg

Ranidu Lankage

Vincent Sung

Yu Wang

Sana Park

Tianyi Sun

Shannon Ward

Tabari Williams

Daniel Rakhamimov

Jian Ren

Pavel Savchenkov

Roman Golobokov

Sergey Smetanin

David Paliwoda

Nikita Demidov

Ivan Babanin

Viktoria Hwang

Arnab Ghosh

Nikita Belosludtcev

Daniel Jonik

Dor Ayalon

Timur Zakirov

Vladimir Gordienko

Aleksandr Zakharov

Sofya Savinova

Dennis Jin

Hao Long

Suraya Shivji

Neil Tagare

Andrew James Mcphee

The subject technology detects a first gesture and a second gesture, each gesture corresponding to an open trigger finger gesture. The subject technology detects a third gesture and a fourth gesture, each gesture corresponding to a closed trigger finger gesture. The subject technology, selects a first virtual object in a first scene. The subject technology detects a first location and a first position of a first representation of a first finger from the third gesture and a second location and a second position of a second representation of a second finger from the fourth gesture. The subject technology detects a first change in the first location and the first position and a second change in the second location and the second position. The subject technology modifies a set of dimensions of the first virtual object to a different set of dimensions.

Virtual object manipulation with gestures in a messaging system

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and a method for rendering a virtual object in a real-world environment depicted in image content based on contextual information. A virtual object template is selected. One or more stylizations for the virtual object template are determined based on contextual information associated with a computing device. A virtual object is generated by applying the one or more stylizations to the virtual object template. The virtual object is rendered within a 3D space captured within a camera feed of the computing device.

Context-based virtual object rendering

Aspects of the present disclosure involve a system comprising a storage medium storing a program and method for rule-based sharing of content collections. The program and method provide for storing, in association with each content collection, a set of rules with first criteria for adding a content item to the content collection, and with second criteria for viewing the content collection; determining, for a first content collection, that the respective first criteria is met for a first user of a first device; providing, based on the determining, for the first user to generate the content item; adding the generated content item to the first content collection; determining, for the first content collection, that the respective second criteria is met for a second user of a second device; and providing, based on the determining, the first content collection to the second device for viewing by the second user.

Sharing of content collections

The subject technology applies, to image data and depth data, a 3D effect including at least one beautification operation based on an augmented reality content generator, the 3D effect including a beautification operation, the beautification operation comprising modifying image data, the image data including a region corresponding to a representation of a face, the beautification operation comprising using a machine learning model for at least one of smoothing blemishes or preserving facial skin texture. The subject technology generates a depth map using at least the depth data. The subject technology generates a segmentation mask based at least on the image data. The subject technology performs background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data. The subject technology generates a 3D message based at least in part on the applied 3D effect including the at least one beautification operation.

Beautification techniques for 3D data in a messaging system

The subject technology detects a location and a position of a representation of a finger in a set of frames captured by a camera of a client device. The subject technology generates a first virtual object based at least in part on the location and the position of the representation of the finger. The subject technology renders the first virtual object within a first scene. The subject technology detects a first collision event corresponding to a first collider of the first virtual object intersecting with a second collider of a second virtual object. The subject technology modifies a set of dimensions of the second virtual object to a second set of dimensions. The subject technology renders the second virtual object based on the second set of dimensions within a second scene. The subject technology provides for display the rendered second virtual object within the second scene.

Cursor functionality for augmented reality content in messaging systems

Systems and methods are provided for performing operations comprising: displaying a plurality of augmented reality painting options; detecting, by a touch input interface of the eyewear device, a first touch input comprising a single finger touching the touch input interface; selecting a first augmented reality painting option of the plurality of augmented reality painting options in response to the first touch input; while continuing to detect continuous touch between the single finger and the touch input interface following selection of the first augmented reality painting option, displaying a second augmented reality painting option related to the first augmented reality painting option; and performing a selection associated with the second augmented reality painting option in response to detecting, by the touch input interface, movement of the single finger along the touch input interface while continuing to detect the continuous touch.

3D painting on an eyewear device

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for rendering three-dimensional captions (3D) in real-world environments depicted in image content. An editing interface is displayed on a client device. The editing interface includes an input component displayed with a view of a camera feed. A first input comprising one or more text characters is received. In response to receiving the first input, a two-dimensional (2D) representation of the one or more text characters is displayed. In response to detecting a second input, a preview interface is displayed. Within the preview interface, a 3D caption based on the one or more text characters is rendered at a position in a 3D space captured within the camera feed. A message is generated that includes the 3D caption rendered at the position in the 3D space captured within the camera feed.

Rendering 3D captions within real-world environments

The subject technology receives frames of a source media content. The subject technology detects from the frames of the source media content, a first gesture indicating a cut point at a particular frame of the source media content, the cut point associated with a trimming operation to be performed on the source media content. The subject technology selects a starting frame and an ending frame from the frames based at least in part on the cut point at the particular frame. The subject technology performs the trimming operation based on the starting frame and the ending frame. The subject technology generates a second media content using the third set of frames. The subject technology provides for display at least a portion of the third set of frames of the second media content.

Auto trimming for augmented reality content in messaging systems

A permission based media system to perform operations that include: presenting a first media object at a client device associated with a user account, the first media object including a reference that identifies the user account; receiving an input that selects the first media object from the client device; determining a permission of the user account based on the reference that identifies the user account; presenting a set of options based on the permission associated with the user account; receiving a selection of an option from among the set of options; and generating a second media object based on the first media object and the selection of the option, according to certain embodiments.

Permission based media composition

The subject technology detects a first gesture corresponding to an open trigger finger gesture. The subject technology detects a location and a position of a representation of a finger from the open trigger finger gesture. The subject technology generates a first virtual object based at least in part on the location and the position of the representation of the finger. The subject technology detects a first collision event. The subject technology detects a second gesture corresponding to a closed trigger finger gesture. The subject technology selects the second virtual object. The subject technology renders the first virtual object as attached to the second virtual object in response to the selecting. The subject technology provides for display the rendered first virtual object as attached to the second virtual object within a first scene.

Trigger gesture for selection of augmented reality content in messaging systems

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for adding time-based captions to captured video. The program and method provide for displaying, by a messaging application, a preview user interface for previewing and editing captured video in order to generate a media content item, the preview user interface including an interface element which is selectable to switch to a captions user interface for adding a caption to the media content item; switching, in response to first user input selecting the interface element, to the captions user interface which is configured to receive user input for caption content and a duration to display the caption content; receiving, via the captions user interface, second user input for the caption content and the duration; and generating the media content item based on the caption content and the duration to display the caption content.

Adding time-based captions to captured video within a messaging system

The subject technology receives a set of frames. The subject technology detect a first gesture correspond to an open trigger finger gesture. The subject technology receives a second set of frames. The subject technology detects from the second set of frames, a second gesture correspond to a closed trigger finger gesture. The subject technology detects a location and a position of a representation of a finger from the closed trigger finger gesture. The subject technology generates a first virtual object based at least in part on the location and the position of the representation of the finger. The subject technology renders a movement of the first virtual object along a vector away from the location and the position of the representation of the finger within a first scene. The subject technology provides for display the rendered movement of the first virtual object along the vector within the first scene.

Shooting interaction using augmented reality content in a messaging system

Eyewear configured to identify movements of a remote physical object to provide a 3D painting interactive augmented reality experience. In an example, the eyewear uses the identified movements of the remote physical object as a virtual brush to create a three-dimensional (3D) virtual object on an eyewear display. The eyewear determines positional information of the remote physical object. The eyewear uses the positional information and responsively displays and edits the virtual object as a function of the relative 3D positioning of the remote physical object.

Augmented reality eyewear 3D painting

The subject technology applies a three-dimensional (3D) effect to image data and depth data based at least in part on an augmented reality content generator. The subject technology generates a segmentation mask based at least on the image data. The subject technology performs background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data. The subject technology generates a packed depth map based at least in part on the a depth map of the depth data. The subject technology generates, using the processor, a message including information related to the applied 3D effect, the image data, and the depth data.

Generating 3D data in a messaging system

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for performing operations comprising: receiving, by a messaging application, a video feed from a camera of a user device that depicts a face; receiving a request to add a 3D caption to the video feed; identifying a graphical element that is associated with context of the 3D caption; and displaying the 3D caption and the identified graphical element in the video feed at a position in 3D space of the video feed proximate to the face depicted in the video feed.

3D captions with semantic graphical elements

Growing community of inventors

Los Angeles, CA, United States of America

Daniel Moreno