Adobe, Inc.

Zhihong Ding

Zhe Lin

Scott David Cohen

Khoi Pham

Jianming Zhang

Jonathan Worthen Brandt

Luis Figueroa

Walter Wei Tuh Chang

Mingyang Ling

Zhifei Zhang

Luis Angel Figueroa

Xiaohui Shen

Michael Kaplan

Qing Liu

Kushal Kafle

Soo Ye Kim

Darshan D Prasad

Brian Lynn Price

Gregg Darryl Wilensky

Walter Chang

Lubomira A Dontcheva

Krishna Kumar Singh

R David Arnold

Quan Hung Tran

Thomas Muehrke

Eric Muller

Matthew Joss

Olena Soroka

Michael Zimmermann

Darshan Prasad

Sangryul Jeon

Celso Gomes

Ohiremen Dibua

Mariette Souppe

Daniil Pakhomov

David Hart

Timothy Wojtaszek

Judy S Lee

Claudia Veronica Roberts

The present disclosure relates to systems, methods, and non-transitory computer-readable media that provides a graphical user interface experience to move objects and generate new shadows within a digital image scene. For instance, in one or more embodiments, the disclosed systems receive a digital image depicting a scene. The disclosed systems receive a selection to position an object in a first location within the scene. Further, the disclosed systems composite an image by placing the object at the first location within the scene of the digital image. Moreover, the disclosed systems generate a modified digital image having a shadow of the object where the shadow is consistent with the scene and provides the modified digital image to the client device.

Moving objects casting a shadow and generating proxy shadows within a digital image

The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For instance, in one or more embodiments, the disclosed systems receive a digital image from a client device. The disclosed systems detect, utilizing a shadow detection neural network, an object portrayed in the digital image. The disclosed systems detect, utilizing the shadow detection neural network, a shadow portrayed in the digital image. The disclosed systems generate, utilizing the shadow detection neural network, an object-shadow pair prediction that associates the shadow with the object.

Detecting shadows and corresponding objects in digital images

The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing to remove a shadow for an object. For instance, in one or more embodiments, the disclosed systems receive a digital image depicting a scene. The disclosed systems access a shadow mask of the shadow in a first location. Further, the disclosed systems generate the modified digital image without the shadow by generating a fill for the first location that preserves a visible location of the first location. Moreover, the disclosed systems generate the digital image without the shadow for the object by combining the fill with the digital image.

Texture-preserving shadow removal in digital images utilizing generating inpainting models

The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For instance, in one or more embodiments, the disclosed systems provide, for display within a graphical user interface of a client device, a digital image displaying a plurality of objects, the plurality of objects comprising a plurality of different types of objects. The disclosed systems generate, utilizing a segmentation neural network and without user input, an object mask for objects of the plurality of objects. The disclosed systems determine, utilizing a distractor detection neural network, a classification for the objects of the plurality of objects. The disclosed systems remove at least one object from the digital image, based on classifying the at least one object as a distracting object, by deleting the object mask for the at least one object.

Removing distracting objects from digital images

The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via multi-layered scene completion techniques facilitated by artificial intelligence. For instance, in some embodiments, the disclosed systems receive a digital image portraying a first object and a second object against a background, where the first object occludes a portion of the second object. Additionally, the disclosed systems pre-process the digital image to generate a first content fill for the portion of the second object occluded by the first object and a second content fill for a portion of the background occluded by the second object. After pre-processing, the disclosed systems detect one or more user interactions to move or delete the first object from the digital image. The disclosed systems further modify the digital image by moving or deleting the first object and exposing the first content fill for the portion of the second object.

Modifying digital images via multi-layered scene completion facilitated by artificial intelligence

The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For instance, in one or more embodiments, the disclosed systems generate utilizing a segmentation neural network, an object mask for each object of a plurality of objects of a digital image. The disclosed systems detect a first user interaction with an object in the digital image displayed via a graphical user interface. The disclosed systems surface, via the graphical user interface, the object mask for the object in response to the first user interaction. The disclosed systems perform an object-aware modification of the digital image in response to a second user interaction with the object mask for the object.

Modifying digital images via scene-based editing using image understanding facilitated by artificial intelligence

Systems and methods for image processing are configured. Embodiments of the present disclosure encode a content image and a style image using a machine learning model to obtain content features and style features, wherein the content image includes a first object having a first appearance attribute and the style image includes a second object having a second appearance attribute; align the content features and the style features to obtain a sparse correspondence map that indicates a correspondence between a sparse set of pixels of the content image and corresponding pixels of the style image; and generate a hybrid image based on the sparse correspondence map, wherein the hybrid image depicts the first object having the second appearance attribute.

Exemplar-based object appearance transfer driven by correspondence

The present disclosure relates to systems, non-transitory computer-readable media, and methods for transferring global style features between digital images utilizing one or more machine learning models or neural networks. In particular, in one or more embodiments, the disclosed systems receive a request to transfer a global style from a source digital image to a target digital image, identify at least one target object within the target digital image, and transfer the global style from the source digital image to the target digital image while maintaining an object style of the at least one target object.

Applying object-aware style transfer to digital images

This disclosure describes one or more implementations of systems, non-transitory computer-readable media, and methods that extract multiple attributes from an object portrayed in a digital image utilizing a multi-attribute contrastive classification neural network. For example, the disclosed systems utilize a multi-attribute contrastive classification neural network that includes an embedding neural network, a localizer neural network, a multi-attention neural network, and a classifier neural network. In some cases, the disclosed systems train the multi-attribute contrastive classification neural network utilizing a multi-attribute, supervised-contrastive loss. In some embodiments, the disclosed systems generate negative attribute training labels for labeled digital images utilizing positive attribute labels that correspond to the labeled digital images.

Extracting attributes from arbitrary digital images utilizing a multi-attribute contrastive classification neural network

The present disclosure relates to a color classification system that accurately classifies objects in digital images based on color. In particular, in one or more embodiments, the color classification system utilizes a multidimensional color space and one or more color mappings to match objects to colors. Indeed, the color classification system can accurately and efficiently detect the color of an object utilizing one or more color similarity regions generated in the multidimensional color space.

Determining colors of objects in digital images

The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For instance, in one or more embodiments, the disclosed systems detect, via a graphical user interface of a client device, a user selection of an object portrayed within a digital image. The disclosed systems determine, in response to detecting the user selection of the object, a relationship between the object and an additional object portrayed within the digital image. The disclosed systems receive one or more user interactions for modifying the object. The disclosed systems modify the digital image in response to the one or more user interactions by modifying the object and the additional object based on the relationship between the object and the additional object.

Detecting object relationships and editing digital images based on the object relationships

The present disclosure relates to a multi-model object segmentation system that provides a multi-model object segmentation framework for automatically segmenting objects in digital images. In one or more implementations, the multi-model object segmentation system utilizes different types of object segmentation models to determine a comprehensive set of object masks for a digital image. In various implementations, the multi-model object segmentation system further improves and refines object masks in the set of object masks utilizing specialized object segmentation models, which results in more improved accuracy and precision with respect to object selection within the digital image. Further, in some implementations, the multi-model object segmentation system generates object masks for portions of a digital image otherwise not captured by various object segmentation models.

Segmenting objects in digital images utilizing a multi-object segmentation model framework

The present disclosure relates to an object selection system that automatically detects and selects objects in a digital image based on natural language-based inputs. For instance, the object selection system can utilize natural language processing tools to detect objects and their corresponding relationships within natural language object selection queries. For example, the object selection system can determine alternative object terms for unrecognized objects in a natural language object selection query. As another example, the object selection system can determine multiple types of relationships between objects in a natural language object selection query and utilize different object relationship models to select the requested query object.

Utilizing natural language processing automatically select objects in images

Systems, methods, and software are described herein for removing people distractors from images. A distractor mitigation solution implemented in one or more computing devices detects people in an image and identifies salient regions in the image. The solution then determines a saliency cue for each person and classifies each person as wanted or as an unwanted distractor based at least on the saliency cue. An unwanted person is then removed from the image or otherwise reduced from the perspective of being an unwanted distraction.

Mitigating people distractors in images

The present disclosure relates to an object selection system that automatically detects and selects objects in a digital image utilizing a large-scale object detector. For instance, in response to receiving a request to automatically select a query object with an unknown object class in a digital image, the object selection system can utilize a large-scale object detector to detect potential objects in the image, filter out one or more potential objects, and label the remaining potential objects in the image to detect the query object. In some implementations, the large-scale object detector utilizes a region proposal model, a concept mask model, and an auto tagging model to automatically detect objects in the digital image.

Growing community of inventors

Fremont, CA, United States of America

Zhihong Ding