Adobe, Inc.

Frederick Mandia

Jianming Zhang

Kalyan Krishna Sunkavalli

Meredith Payne Stotzner

Zhe Lin

Elya Shechtman

Jose Ignacio Echevarria Vallespi

Sarah Kong

He Zhang

I-Ming Pao

Alan Lee Erickson

Hyunghwan Byun

Yinglan Ma

Betty Leong

Yukie Takahashi

Guotong Feng

Quynn Megan Le

The present disclosure relates to systems, non-transitory computer-readable media, and methods that implement a dual-branched neural network architecture to harmonize composite images. For example, in one or more implementations, the transformer-based harmonization system uses a convolutional branch and a transformer branch to generate a harmonized composite image based on an input composite image and a corresponding segmentation mask. More particularly, the convolutional branch comprises a series of convolutional neural network layers followed by a style normalization layer to extract localized information from the input composite image. Further, the transformer branch comprises a series of transformer neural network layers to extract global information based on different resolutions of the input composite image. Utilizing a decoder, the transformer-based harmonization system combines the local information and the global information from the corresponding convolutional branch and transformer branch to generate a harmonized composite image.

Harmonizing composite images utilizing a transformer neural network

The present disclosure provides systems and methods for image editing. Embodiments of the present disclosure provide an image editing system for perform image object replacement or image region replacement (e.g., an image editing system for replacing an object or region of an image with an object or region from another image). For example, the image editing system may replace a sky portion of an image with a more desirable sky portion from a different replacement image. The original image and the replacement image (e.g., the image including a desirable object or region) include layers of masks. A sky from the replacement image may replace the sky of the image to produce an aesthetically pleasing composite image.

Sky replacement

Growing community of inventors

San Jose, CA, United States of America

Frederick Mandia