Adobe, Inc.

Scansoft, Inc.

Su Chen

Jianming Zhang

Zhe Lin

Simon Niklaus

Kalyan Krishna Sunkavalli

Oliver Wang

Sohrab Amirghodsi

Yilin Wang

Sarah Kong

He Zhang

Mai Long

Yifan Jiang

Wei Yin

Scott David Cohen

Brian Lynn Price

Barnaby John James

Zijun Wei

Jason Wen Yong Kuen

Soo Ye Kim

Shuo Yang

Yifei Fan

The present disclosure relates to systems, non-transitory computer-readable media, and methods for utilizing machine learning models to generate refined depth maps of digital images utilizing digital segmentation masks. In particular, in one or more embodiments, the disclosed systems generate a depth map for a digital image utilizing a depth estimation machine learning model, determine a digital segmentation mask for the digital image, and generate a refined depth map from the depth map and the digital segmentation mask utilizing a depth refinement machine learning model. In some embodiments, the disclosed systems generate first and second intermediate depth maps using the digital segmentation mask and an inverse digital segmentation mask and merger the first and second intermediate depth maps to generate the refined depth map.

Utilizing machine learning models to generate refined depth maps with segmentation mask guidance

The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately, efficiently, and flexibly generating harmonized digital images utilizing a self-supervised image harmonization neural network. In particular, the disclosed systems can implement, and learn parameters for, a self-supervised image harmonization neural network to extract content from one digital image (disentangled from its appearance) and appearance from another from another digital image (disentangled from its content). For example, the disclosed systems can utilize a dual data augmentation method to generate diverse triplets for parameter learning (including input digital images, reference digital images, and pseudo ground truth digital images), via cropping a digital image with perturbations using three-dimensional color lookup tables ('LUTs'). Additionally, the disclosed systems can utilize the self-supervised image harmonization neural network to generate harmonized digital images that depict content from one digital image having the appearance of another digital image.

Learning parameters for an image harmonization neural network to generate deep harmonized digital images

The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object masks for digital objects portrayed in digital images utilizing a detection-masking neural network pipeline. In particular, in one or more embodiments, the disclosed systems utilize detection heads of a neural network to detect digital objects portrayed within a digital image. In some cases, each detection head is associated with one or more digital object classes that are not associated with the other detection heads. Further, in some cases, the detection heads implement multi-scale synchronized batch normalization to normalize feature maps across various feature levels. The disclosed systems further utilize a masking head of the neural network to generate one or more object masks for the detected digital objects. In some cases, the disclosed systems utilize post-processing techniques to filter out low-quality masks.

Detecting digital objects and generating object masks on device

Generating deep harmonized digital images

This disclosure describes one or more implementations of a depth prediction system that generates accurate depth images from single input digital images. In one or more implementations, the depth prediction system enforces different sets of loss functions across mix-data sources to generate a multi-branch architecture depth prediction model. For instance, in one or more implementations, the depth prediction model utilizes different data sources having different granularities of ground truth depth data to robustly train a depth prediction model. Further, given the different ground truth depth data granularities from the different data sources, the depth prediction model enforces different combinations of loss functions including an image-level normalized regression loss function and/or a pair-wise normal loss among other loss functions.

Generating depth images utilizing a machine-learning model built from mixed digital image sources and multiple loss function sets

The present disclosure relates to systems, non-transitory computer-readable media, and methods that utilize a deep neural network to process object user indicators and an initial object segmentation from a digital image to efficiently and flexibly generate accurate object segmentations. In particular, the disclosed systems can determine an initial object segmentation for the digital image (e.g., utilizing an object segmentation model or interactive selection processes). In addition, the disclosed systems can identify an object user indicator for correcting the initial object segmentation and generate a distance map reflecting distances between pixels of the digital image and the object user indicator. The disclosed systems can generate an image-interaction-segmentation triplet by combining the digital image, the initial object segmentation, and the distance map. By processing the image-interaction-segmentation triplet utilizing the segmentation neural network, the disclosed systems can provide an updated object segmentation for display to a client device.

Utilizing a segmentation neural network to process initial object segmentations and object user indicators within a digital image to generate improved object segmentations

This disclosure describes implementations of a three-dimensional (3D) scene recovery system that reconstructs a 3D scene representation of a scene portrayed in a single digital image. For instance, the 3D scene recovery system trains and utilizes a 3D point cloud model to recover accurate intrinsic camera parameters from a depth map of the digital image. Additionally, the 3D point cloud model may include multiple neural networks that target specific intrinsic camera parameters. For example, the 3D point cloud model may include a depth 3D point cloud neural network that recovers the depth shift as well as include a focal length 3D point cloud neural network that recovers the camera focal length. Further, the 3D scene recovery system may utilize the recovered intrinsic camera parameters to transform the single digital image into an accurate and realistic 3D scene representation, such as a 3D point cloud.

Reconstructing three-dimensional scenes portrayed in digital images utilizing point cloud machine-learning models

The present invention is a system and method for performing document recognition and processing in a distributed computing environment. The invention uses applications which are remotely located from one or more users and may be accessed via a network. One or more users utilize terminals including computers, facsimile machines, and/or scanners to transmit documents to be processed to a network or a network server which in turn transmits the documents to various computer software applications which process the documents at a network processing location. Once the documents have been processed, the processed documents are transmitted to the users according to one or more preferences associated with a user identification and/or authentication which may be determined by either a network server or an application server. Users utilizing a computer terminal make use of various data transfer programs capable of transferring document data over a network to an application server at a remote location and receiving processed document data via a network.

Distributed computing document recognition and processing

Growing community of inventors

San Jose, CA, United States of America

Su Chen