Adobe, Inc.

Zichuan Liu

Xin Lu

Jen-Chan Jeff Chien

Wentian Zhao

Kun Wan

Yumin Jia

Yeojin Kim

Shitong Wang

He Qin

Sheng-Wei Huang

Ke Wang

Mingyuan Wu

In implementations of systems for efficient object segmentation, a computing device implements a segment system to receive a user input specifying coordinates of a digital image. The segment system computes receptive fields of a machine learning model based on the coordinates of the digital image. The machine learning model is trained on training data to generate segment masks for objects depicted in digital images. The segment system processes a portion of a feature map of the digital image using the machine learning model based on the receptive fields. A segment mask is generated for an object depicted in the digital image based on processing the portion of the feature map of the digital image using the machine learning model.

Efficient object segmentation

A method for performing efficient mixed-precision search for an artificial neural network (ANN) includes training the ANN by sampling selected candidate quantizers of a bank of candidate quantizer and updating network parameters for a next iteration based on outputs of layers of the ANN. The outputs are computed by processing quantized data with operators (e.g., convolution). The quantizers converge to optimal bit-widths that reduce classification losses bounded by complexity constrains.

Efficient mixed-precision search for quantizers in artificial neural networks

Methods, systems, and non-transitory computer readable storage media are disclosed for generating image mattes for detected objects in digital images without trimap segmentation via a multi-branch neural network. The disclosed system utilizes a first neural network branch of a generative neural network to extract a coarse semantic mask from a digital image. The disclosed system utilizes a second neural network branch of the generative neural network to extract a detail mask based on the coarse semantic mask. Additionally, the disclosed system utilizes a third neural network branch of the generative neural network to fuse the coarse semantic mask and the detail mask to generate an image matte. In one or more embodiments, the disclosed system also utilizes a refinement neural network to generate a final image matte by refining selected portions of the image matte generated by the generative neural network.

Generating image mattes without trimap segmentations via a multi-branch neural network

The present disclosure describes systems, non-transitory computer-readable media, and methods for accurately and efficiently removing objects from digital images taken from a camera viewfinder stream. For example, the disclosed systems access digital images from a camera viewfinder stream in connection with an undesired moving object depicted in the digital images. The disclosed systems generate a temporal window of the digital images concatenated with binary masks indicating the undesired moving object in each digital image. The disclosed systems further utilizes a generator as part of a 3D to 2D generative adversarial neural network in connection with the temporal window to generate a target digital image with the region associated with the undesired moving object in-painted. In at least one embodiment, the disclosed systems provide the target digital image to a camera viewfinder display to show a user how a future digital photograph will look without the undesired moving object.

Automatically removing moving objects from video streams

The present disclosure relates to systems, methods, and non-transitory computer-readable media that generate refined segmentation masks for digital visual media items. For example, in one or more embodiments, the disclosed systems utilize a segmentation refinement neural network to generate an initial segmentation mask for a digital visual media item. The disclosed systems further utilize the segmentation refinement neural network to generate one or more refined segmentation masks based on uncertainly classified pixels identified from the initial segmentation mask. To illustrate, in some implementations, the disclosed systems utilize the segmentation refinement neural network to redetermine whether a set of uncertain pixels corresponds to one or more objects depicted in the digital visual media item based on low-level (e.g., local) feature values extracted from feature maps generated for the digital visual media item.

Iteratively refining segmentation masks

The present disclosure describes systems, non-transitory computer-readable media, and methods for accurately and efficiently removing objects from digital images taken from a camera viewfinder stream. For example, the disclosed systems access digital images from a camera viewfinder stream in connection with an undesired moving object depicted in the digital images. The disclosed systems generate a temporal window of the digital images concatenated with binary masks indicating the undesired moving object in each digital image. The disclosed systems further utilizes a 3D to 2D generator as part of a 3D to 2D generative adversarial neural network in connection with the temporal window to generate a target digital image with the region associated with the undesired moving object in-painted. In at least one embodiment, the disclosed systems provide the target digital image to a camera viewfinder display to show a user how a future digital photograph will look without the undesired moving object.

Growing community of inventors

San Jose, CA, United States of America

Zichuan Liu