L'oreal

Brendan Duke

Parham Aarabi

Irina Kezele

Abdalla Ahmed

Ruowei Jiang

Edmund Phung

Rohit Saha

Cong Wei

Vision Transformers (ViT) have shown their competitive advantages performance-wise compared to convolutional neural networks (CNNs) though they often come with high computational costs. Methods, systems and techniques herein learn instance-dependent attention patterns, utilizing a lightweight connectivity predictor module to estimate a connectivity score of each pair of tokens. Intuitively, two tokens have high connectivity scores if the features are considered relevant either spatially or semantically. As each token only attends to a small number of other tokens, the binarized connectivity masks are often very sparse by nature providing an opportunity to accelerate the network via sparse computations. Equipped with the learned unstructured attention pattern, sparse attention ViT produces a superior Pareto-optimal trade-off between FLOPs and top-1 accuracy on ImageNet compared to token sparsity (48%˜69% FLOPs reduction of MHSA; accuracy drop within 0.4%). Combining attention and token sparsity reduces VIT FLOPs by over 60%.

Systems, methods and techniques for learning and using sparse instance-dependent attention for efficient vision transformers

Methods, apparatus and technique embodiments localize objects using antialiasing such as for rendering an object with an effect for a virtual try on (VTO) experience. An example system comprises a nail localization engine including computational circuitry to localize one or more nail objects in an input image of a hand or foot via one or more deep neural networks, wherein the one or more deep neural networks is configured for antialiasing in each of the encoder and decoder components; and a rendering component including computational circuitry to render an output image simulating a nail product or nail service applied to the one or more nail objects, responsive to the localizing by the nail localization engine, to provide a virtual try on (VTO) experience. In an embodiment, respective blur operators in each of the encoders and decoders disperse values to neighboring regions to counteract shift variance.

Method and system for object antialiasing in an augmented reality experience

Hairstyle transfer is challenging due to hair structure differences in the source and target hair. Latent Optimization of Hairstyles via Orthogonalization (LOHO) is an optimization-based approach using GAN inversion to infill missing hair structure details in latent space during hairstyle transfer. Hair is decomposed into three attributes: perceptual structure, appearance, and finer style, and includes tailored losses to model each of these attributes independently. Two-stage optimization and gradient orthogonalization enable disentangled latent space optimization of the three hair attributes. Using LOHO for latent space manipulation, users can synthesize novel photorealistic images by manipulating hair attributes either individually or jointly, transferring the desired attributes from reference hairstyles.

Style try-on with reverse GANs

There are provided methods and computing devices using semi-supervised learning to perform end-to-end video object segmentation, tracking respective object(s) from a single-frame annotation of a reference frame through a video sequence of frames. A known deep learning model may be used to annotate the reference frame to provide ground truth locations and masks for each respective object. A current frame is processed to determine current frame object locations, defining object scoremaps as a normalized cross-correlation between encoded object features of the current frame and encoded object features of a previous frame. Scoremaps for each of more than one previous frame may be defined. An Intersection over Union (IoU) function, responsive to the scoremaps, ranks candidate object proposals defined from the reference frame annotation to associate the respective objects to respective locations in the current frame. Pixel-wise overlap may be removed using a merge function responsive to the scoremaps.

End-to-end merge for video object segmentation (VOS)

Presented is a convolutional neural network (CNN) model for fingernail tracking, and a method design for nail polish rendering. Using current software and hardware, the CNN model and method to render nail polish runs in real-time on both iOS and web platforms. A use of Loss Mean Pooling (LMP) coupled with a cascaded model architecture simultaneously enables pixel-accurate fingernail predictions at up to 640&#xd7;480 resolution. The proposed post-processing and rendering method takes advantage of the model's multiple output predictions to render gradients on individual fingernails, and to hide the light-colored distal edge when rendering on top of natural fingernails by stretching the nail mask in the direction of the fingernail tip. Teachings herein may be applied to track objects other than fingernails and to apply appearance effects other than color.

Growing community of inventors

Toronto, Canada

Brendan Duke