Snap Inc.

Ning Zhang

Jia Li

Yuncheng Li

Linjie Luo

Rahul Bhupendra Sheth

Jianchao Yang

Xiaoyu Wang

Drake Austin Rehfeld

Shenlong Wang

Xuecheng Nie

Ning Xu

Samuel Edward Hare

Tony Mathew

Kevin Dechau Tang

Zhou Ren

Linjie Yang

Leonardo Ribas Machado Das Neves

Jonathan Solichin

Xutao Lv

Travis Chen

Vitor R Carvalho

Vitor Rocha De Carvalho

Zhengyuan Yang

Di Lu

Wei Han

Jacob Minyoung Huh

Shao-Hua Sun

Segmentation of an image into individual body parts is performed based on a trained model. The model is trained with a plurality of training images, each training image representing a corresponding training figure. The model is also trained with a corresponding plurality of segmentations of the training figures. Each segmentation is generated by positioning body parts between defined positions of joints of the represented figure. The body parts are represented by body part templates obtained from a template library, with the templates defining characteristics of body parts represented by the templates.

Weakly supervised semantic parsing

An example system includes an encoder configured to receive a bit string and encode the bit string into a visual representation, and a decoder configured to receive an image including the visual representation and decode the bit string from the visual representation. In some examples, the encoder and decoder are trained as a pair by obtaining a training bit string, encoding the training bit string into a training visual representation using the encoder, decoding the training visual representation using the decoder to generate a decoded bit string, determining an error between the training bit string and the decoded bit string, and updating parameters of the encoder and decoder to reduce the error.

Encoding and decoding a stylized custom graphic

Dense feature scale detection can be implemented using multiple convolutional neural networks trained on scale data to more accurately and efficiently match pixels between images. An input image can be used to generate multiple scaled images. The multiple scaled images are input into a feature net, which outputs feature data for the multiple scaled images. An attention net is used to generate an attention map from the input image. The attention map assigns emphasis as a soft distribution to different scales based on texture analysis. The feature data and the attention data can be combined through a multiplication process and then summed to generate dense features for comparison.

Dense feature scale detection for image matching

Systems, devices, media and methods are presented for a human pose tracking framework. The human pose tracking framework may identify a message with video frames, generate, using a composite convolutional neural network, joint data representing joint locations of a human depicted in the video frames, the generating of the joint data by the composite convolutional neural network done by a deep convolutional neural network operating on one portion of the video frames, a shallow convolutional neural network operating on a another portion of the video frames, and tracking the joint locations using a one-shot learner neural network that is trained to track the joint locations based on a concatenation of feature maps and a convolutional pose machine. The human pose tracking framework may store, the joint locations, and cause presentation of a rendition of the joint locations on a user interface of a client device.

Efficient human pose tracking in videos

A caption of a multimodal message (e.g., social media post) can be identified as a named entity using an entity recognition system. The entity recognition system can use a visual attention based mechanism to generate a visual context representation from an image and caption. The system can use the visual context representation to identify one or more terms of the caption as a named entity.

Named entity recognition visual context and caption data

A system according to various exemplary embodiments includes a processor and a user interface, communication module, and memory coupled to the processor. The memory stores instructions that, when executed by the processor, cause the system to: retrieve a digital image from a server using the communication module; present the digital image on a display of the user interface; receive edits to the digital image via the user interface; generate, based on the edits, a modified digital image, wherein generating the modified digital image includes transforming a format of the digital image to include a field containing an identifier associated with the modified digital image; and transmit the modified digital image to the server using the communication module.

Systems and methods for digital image editing

Disclosed are methods for encoding information in a graphic image. The information may be encoded so as to have a visual appearance that adopts a particular style, so that the encoded information is visually pleasing in the environment in which it is displayed. An encoder and decoder are trained during an integrated training process, where the encoder is tuned to minimize a loss when its encoded images are decoded. Similarly, the decoder is also trained to minimize loss when decoding the encoded images. Both the encoder and decoder may utilize a convolutional neural network in some aspects to analyze data and/or images. Once data is encoded, a style from a sample image is transferred to the encoded data. When decoding, the decoder may largely ignore the style aspects of the encoded data and decode based on a content portion of the data.

Systems, devices, media, and methods are presented for object detection and inserting graphical elements into an image stream in response to detecting the object. The systems and methods detect an object of interest in received frames of a video stream. The systems and methods identify a bounding box for the object of interest and estimate a three-dimensional position of the object of interest based on a scale of the object of interest. The systems and methods generate one or more graphical elements having a size based on the scale of the object of interest and a position based on the three-dimensional position estimated for the object of interest. The one or more graphical elements are generated within the video stream to form a modified video stream. The systems and methods cause presentation of the modified video stream including the object of interest and the one or more graphical elements.

Neural network-based image stream modification

Systems, methods, devices, media, and computer readable instructions are described for local image tagging in a resource constrained environment. One embodiment involves processing image data using a deep convolutional neural network (DCNN) comprising at least a first subgraph and a second subgraph, the first subgraph comprising at least a first layer and a second layer, processing, the image data using at least the first layer of the first subgraph to generate first intermediate output data; processing, by the mobile device, the first intermediate output data using at least the second layer of the first subgraph to generate first subgraph output data, and in response to a determination that each layer reliant on the first intermediate data have completed processing, deleting the first intermediate data from the mobile device. Additional embodiments involve convolving entire pixel resolutions of the image data against kernels in different layers if the DCNN.

Content tagging

Systems, devices, media, and methods are presented for identifying and categorically labeling objects within a set of images. The systems and methods receive an image depicting an object of interest, detect at least a portion of the object of interest within the image using a multilayer object model, determine context information, and identify the object of interest included in two or more bounding boxes.

Growing community of inventors

Los Angeles, CA, United States of America

Ning Zhang