Snap Inc.

Lemon Inc.

Adobe, Inc.

Bytedance Inc.

Beijing Bytedance Network Technology Co., Ltd.

Beijing Zitiao Network Technology Co., Ltd.

Linjie Luo

Jia Li

Rahul Bhupendra Sheth

Jing Liu

Jonathan M Rodriguez, Ii

Xutao Lv

Nathan Jurgenson

Jianchao Yang

Menglei Chai

Ning Zhang

Chen Cao

Guohui Wang

Chunpong Lai

Wanchun Ma

Guoxian Song

Ning Xu

Shenlong Wang

Zhili Chen

Yuncheng Li

Chongyang Ma

Xiao Yang

Tiancheng Zhi

Xuecheng Nie

Zehao Xue

Oliver Woodford

Jingcong Zhang

Shen Sang

Shuo Cheng

Soumyadip Sengupta

Jonathan Worthen Brandt

Nathan Aaron Carr

Xu Wang

Daniel Goldman

Wilmot Wei-Mau Li

Yunzhu Li

Xuchen Song

Jingna Sun

Weihong Zeng

Haoze Li

Michael Leong Hou Tay

Hongyi Xu

Xiang Yu

Jan Jachnik

Wen Zhang

Chao Wang

Kalyan Krishna Sunkavalli

Sunil Sharadchandra Hadap

Byungmoon Kim

Nan Hu

Kun Duan

Tao Hu

Ye Yuan

Jiashi Feng

Xiaochen Lian

Yuanlong Chen

Beixin Hu

Jianfeng Zhang

Rushikesh Dudhat

Zihang Jiang

Yichun Shi

Jonathan M Ii Rodriguez

Jia Lia

A positioning model optimization method, a positioning method, and a positioning device are provided. The positioning model optimization method includes: inputting a positioning model for a scene, the positioning model including a three-dimensional (3D) point cloud and a plurality of descriptors corresponding to each 3D point in the 3D point cloud; calculating a significance of each 3D point in the 3D point cloud, and if the significance is greater than a predetermined threshold, outputting the 3D point and the plurality of descriptors corresponding to the 3D point to an optimized positioning model for the scene; and outputting the optimized positioning model for the scene.

Positioning model optimization method, positioning method, and positioning device

An animation processing method and apparatus, an electronic device and a storage medium, applied to a shader, the method including: acquiring an animation sample of a model unit; acquiring an external interaction parameter of an augmented reality model, where the augmented reality model includes a plurality of model units; and outputting the augmented reality model, and driving the animation sample of the model unit in the output augmented reality model according to the external interaction parameter. The animation sample of the model unit can be driven according to the external interaction parameter, so that animation of the model can be adjusted according to user operation, thereby improving usability.

Animation processing method and apparatus, electronic device and storage medium

Technologies are described and recited herein for producing controllable synthesized images include a geometry guided 3D GAN framework for high-quality 3D head synthesis with full control on camera poses, facial expressions, head shape, articulated neck and jaw poses; and a semantic SDF (signed distance function) formulation that defines volumetric correspondence from observation space to canonical space, allowing full disentanglement of control parameters in 3D GAN training.

Geometry-guided controllable 3D head synthesis

An image effect implementing method is described herein that includes: acquiring a first image, recognizing a set object in the first image, and acquiring an augmented reality model corresponding to the set object; superimposing, according to coordinate information of pixels of the set object, the augmented reality model onto the first image to obtain a second image; and upon detection of a preset deformation event, controlling, according to a set deformation policy, at least one sub-model of the augmented reality model in the second image to deform, and displaying the deformed second image.

Image effect implementing method and apparatus, electronic device and storage medium

Three-dimensional (3D) avatars may be produced by stylizing a dataset of images based on a user-input text prompt input to a stable diffusion model, and using the output stylized dataset of images to train an efficient geometry-aware 3D generative adversarial network (EG3D) model.

Text-to-3D avatars

A method of generating a stylized 3D avatar is provided. The method includes receiving an input image of a user, generating, using a generative adversarial network (GAN) generator, a stylized image, based on the input image, and providing the stylized image to a first model to generate a first plurality of parameters. The first plurality of parameters include a discrete parameter and a continuous parameter. The method further includes providing the stylized image and the first plurality of parameters to a second model that is trained to generate an avatar image, receiving, from the second model, the avatar image, comparing the stylized image to the avatar image, based on a loss function, to determine an error, updating the first model to generate a second plurality of parameters that correspond to the first plurality of parameters, based on the error, and providing the second plurality of parameters as an output.

Cascaded domain bridging for image generation

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and a method for accessing a set of images depicting at least a portion of a face. A set of facial regions of the face is identified, each facial region of the set of facial regions intersecting another facial region with at least one common vertex that is a member of a set of facial vertices. For each facial region of the set of facial regions, a weight formed from a set of region coefficients is generated. Based on the set of facial regions and the weight of each facial region of the set of facial regions, the face is tracked across the set of images.

Region-based stabilized face tracking

A method of generating a style image is described. The method includes receiving an input image of a subject. The method further includes encoding the input image using a first encoder of a generative adversarial network (GAN) to obtain a first latent code. The method further includes decoding the first latent code using a first decoder of the GAN to obtain a normalized style image of the subject, wherein the GAN is trained using a loss function according to semantic regions of the input image and the normalized style image.

Systems and methods for multi-task joint training of a neural network including an encoder module and a multi-headed attention mechanism are provided. In one aspect, the system includes a processor configured to receive input data including a first set of labels and a second set of labels. Using the encoder module, features are extracted from the input data. Using a multi-headed attention mechanism, training loss metrics are computed. A first training loss metric is computed using the extracted features and the first set of labels, and a second training loss metric is computed using the extracted features and the second set of labels. A first mask is applied to filter the first training loss metric, and a second mask is applied to filter the second training loss metric. A final training loss metric is computed based on the filtered first and second training loss metrics.

Systems for multi-task joint training of neural networks using multi-label datasets

A dolly zoom effect can be applied to one or more images captured via a resource-constrained device (e.g., a mobile smartphone) by manipulating the size of a target feature while the background in the one or more images changes due to physical movement of the resource-constrained device. The target feature can be detected using facial recognition or shape detection techniques. The target feature can be resized before the size is manipulated as the background changes (e.g., changes perspective).

Scaled perspective zoom on resource constrained devices

Systems and methods directed to controlling the similarity between stylized portraits and an original photo are described. In examples, an input image is received and encoded using a variational autoencoder to generate a latent vector. The latent vector may be blended with latent vectors that best represent a face in the original user portrait image. The resulting blended latent vector may be provided to a generative adversarial network (GAN) generator to generate a controlled stylized image. In examples, one or more layers of the stylized GAN generator may be swapped with one or more layers of the original GAN generator. Accordingly, a user can interactively determine how much stylization vs. personalization should be included in a resulting stylized portrait.

Portrait stylization framework to control the similarity between stylized portraits and original photo

Dense feature scale detection can be implemented using multiple convolutional neural networks trained on scale data to more accurately and efficiently match pixels between images. An input image can be used to generate multiple scaled images. The multiple scaled images are input into a feature net, which outputs feature data for the multiple scaled images. An attention net is used to generate an attention map from the input image. The attention map assigns emphasis as a soft distribution to different scales based on texture analysis. The feature data and the attention data can be combined through a multiplication process and then summed to generate dense features for comparison.

Dense feature scale detection for image matching

Methods and systems for enlarging a stylized region of an image are disclosed that include receiving an input image, generating, using a first generative adversarial network (GAN) generator, a first stylized image, based on the input image, normalizing the input image, generating, using a second generative adversarial network (GAN) generator, a second stylized image, based on the normalized input image, blending the first stylized image and the second stylized image to obtain a third stylized image, and providing the third stylized image as an output.

Agilegan-based stylization method to enlarge a style region

Methods and systems for generating a texturized image are disclosed. Some examples may include: receiving an input image, receiving an exemplar texture image, generating, using an encoder, a first latent code vector representation based on the input image, generating, using a generative adversarial network generator, a second latent code vector representation based on the exemplar texture image, blending the first latent code vector representation and the second latent code vector representation to obtain a blended latent code vector representation, generating, by the GAN generator, a texturized image based on the blended latent code vector representation and providing the texturized image as an output image.

Agilegan-based refinement method and framework for consistent texture generation

Systems, devices, media and methods are presented for a human pose tracking framework. The human pose tracking framework may identify a message with video frames, generate, using a composite convolutional neural network, joint data representing joint locations of a human depicted in the video frames, the generating of the joint data by the composite convolutional neural network done by a deep convolutional neural network operating on one portion of the video frames, a shallow convolutional neural network operating on a another portion of the video frames, and tracking the joint locations using a one-shot learner neural network that is trained to track the joint locations based on a concatenation of feature maps and a convolutional pose machine. The human pose tracking framework may store, the joint locations, and cause presentation of a rendition of the joint locations on a user interface of a client device.

Efficient human pose tracking in videos

Growing community of inventors

Los Angeles, CA, United States of America

Linjie Luo