Adobe, Inc.

Splunk Inc.

Jianming Zhang

Zhe Lin

Xiaohui Shen

Yilin Wang

Scott David Cohen

He Zhang

Radomir Mech

Elya Shechtman

Jingwan Lu

Ning Xu

Qing Liu

Zijun Wei

Sohrab Amirghodsi

Haitian Zheng

Brian Lynn Price

Jonathan Worthen Brandt

Jimei Yang

Mingyang Ling

Jason Wen Yong Kuen

Oliver Wang

Federico Perazzi

Su Chen

Kalyan Krishna Sunkavalli

Zhihong Ding

Jen-Chan Jeff Chien

Baldo Antonio Faieta

Connelly Barnes

Krishna Kumar Singh

Lu Zhang

Jose Ignacio Echevarria Vallespi

Chen Fang

Xin Sun

Sarah Kong

Joon-Young Lee

Anant Gilra

Alex Charles Filipkowski

Seyed Morteza Safdarnejad

Simon Niklaus

Mai Long

Brett Michael Butterfield

Soo Ye Kim

Salil Tambe

Siyuan Qiao

Jason Kuen

Chenglin Yang

Shikun Liu

Yu Zeng

Hailin Jin

Steve Yu Zhang

Xin Lu

Alexandros Batsakis

Quan Anh Tran

Duygu Ceylan Aksit

Yijun Li

Handong Zhao

Yannick Hold-Geoffroy

I-Ming Pao

Yangtuanfeng Wang

Ruben Eduardo Villegas

Balaji Rao

Samarth Gulati

Yinglan Ma

Tobias Hinz

Lijun Wang

Luis Figueroa

Betty Leong

Yingwei Li

Matthew Joss

Yifan Jiang

Wei Yin

Ajinkya Gorakhnath Kale

Aaron Phillip Hertzmann

Dingzeyu Li

Jayant Kumar

Ratheesh Kalarot

John Philip Collomosse

Alan Lee Erickson

Zhifei Zhang

Sanyam Jain

Yuqian Zhou

Midhun Harikumar

Haoxiang Li

Akhilesh Kumar

Daniel Miranda

Chenxi Liu

Hyunghwan Byun

Pramod Srinivasan

Yufei Wang

Kerem Can Turgutlu

Amish Kumar Bedi

Klaas Stöeckmann

Nitilaksha Halakatti

Sarah Stuckey

Julien Philip

Yukie Takahashi

Manuel Martinez

Ningxuan He

Jinrong Xie

Jean-François Lalonde

Embodiments of the present disclosure include obtaining a text prompt describing an element, layout information indicating a target region for the element, and a precision level corresponding to the element. Some embodiments generate a text feature pyramid based on the text prompt, the layout information, and the precision level, wherein the text feature pyramid comprises a plurality of text feature maps at a plurality of scales, respectively. Then, an image is generated based on the text feature pyramid. In some cases, the image includes an object corresponding to the element of the text prompt at the target region. Additionally, a shape of the object corresponds to a shape of the target region based on the precision level.

Semantic image synthesis

The present disclosure relates to systems, methods, and non-transitory computer-readable media that implement perspective-aware object move operations for digital image editing. For instance, in some embodiments, the disclosed systems determine a vanishing point associated with a digital image portraying an object. Additionally, the disclosed systems detect one or more user interactions for moving the object within the digital image. Based on moving the object with respect to the vanishing point, the disclosed systems perform a perspective-based resizing of the object within the digital image.

Modifying digital images via perspective-aware object move

The present disclosure relates to systems, methods, and non-transitory computer readable media for panoptically guiding digital image inpainting utilizing a panoptic inpainting neural network. In some embodiments, the disclosed systems utilize a panoptic inpainting neural network to generate an inpainted digital image according to panoptic segmentation map that defines pixel regions corresponding to different panoptic labels. In some cases, the disclosed systems train a neural network utilizing a semantic discriminator that facilitates generation of digital images that are realistic while also conforming to a semantic segmentation. The disclosed systems generate and provide a panoptic inpainting interface to facilitate user interaction for inpainting digital images. In certain embodiments, the disclosed systems iteratively update an inpainted digital image based on changes to a panoptic segmentation map.

Panoptically guided inpainting utilizing a panoptic inpainting neural network

In various examples, a set of camera parameters associated with an input image are determined based on a disparity map and a signed defocus map. For example, a disparity model generates the disparity map indicating disparity values associated with pixels of the input image and a defocus model generates a signed defocus map indicating blur values associated with the pixels of the input image.

Single image camera parameter estimation

The present disclosure relates to systems, methods, and non-transitory computer-readable media that provides a graphical user interface experience to move objects and generate new shadows within a digital image scene. For instance, in one or more embodiments, the disclosed systems receive a digital image depicting a scene. The disclosed systems receive a selection to position an object in a first location within the scene. Further, the disclosed systems composite an image by placing the object at the first location within the scene of the digital image. Moreover, the disclosed systems generate a modified digital image having a shadow of the object where the shadow is consistent with the scene and provides the modified digital image to the client device.

Moving objects casting a shadow and generating proxy shadows within a digital image

Systems and methods for instance segmentation are described. Embodiments include identifying an input image comprising an object that includes a visible region and an occluded region that is concealed in the input image. A mask network generates an instance mask for the input image that indicates the visible region of the object. A diffusion model then generates a segmentation mask for the input image based on the instance mask. The segmentation mask indicates a completed region of the object that includes the visible region and the occluded region.

Amodal instance segmentation using diffusion models

The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing to remove a shadow for an object. For instance, in one or more embodiments, the disclosed systems receive a digital image depicting a scene. The disclosed systems access a shadow mask of the shadow in a first location. Further, the disclosed systems generate the modified digital image without the shadow by generating a fill for the first location that preserves a visible location of the first location. Moreover, the disclosed systems generate the digital image without the shadow for the object by combining the fill with the digital image.

Texture-preserving shadow removal in digital images utilizing generating inpainting models

In implementations of systems for performing multiple segmentation tasks, a computing device implements a segment system to receive input data describing a digital image depicting an object. The segment system computes per-pixel embeddings for the digital image using a pixel decoder of a machine learning model. Output embeddings are generated using a transformer decoder of the machine learning model based on the per-pixel embeddings for the digital image, input embeddings for a first segmentation task and input embeddings for a second segmentation task. The segment system outputs a first digital image and a second digital image. The first digital image depicts the object segmented based on the first segmentation task and the second digital image depicts the object segmented based on the second segmentation task.

Performing multiple segmentation tasks

A computing device can receive a query that identifies a set of data to be processed and determine that a portion of the set of data resides in an external data system. The query system can request data identifiers associated with data objects of the set of data from the external data system and communicate the data identifiers to a data queue. The computing device can instruct one or more search nodes to retrieve the identifiers from the data queue. The search nodes can use the data identifiers to retrieve data objects from the external data system and process the data objects according to instructions received from the computing device. The search nodes can provide results of the processing to the computing device.

Retrieving data identifiers from queue for search of external data system

Systems and methods for image processing are described. Embodiments of the present disclosure receive an image comprising a first region that includes content and a second region to be inpainted. Noise is then added to the image to obtain a noisy image, and a plurality of intermediate output images are generated based on the noisy image using a diffusion model trained using a perceptual loss. The intermediate output images predict a final output image based on a corresponding intermediate noise level of the diffusion model. The diffusion model then generates the final output image based on the intermediate output image. The final output image includes inpainted content in the second region that is consistent with the content in the first region.

Image and object inpainting with diffusion models

The present disclosure relates to systems, methods, and non-transitory computer readable media that generate inpainted digital images utilizing class-specific cascaded modulation inpainting neural network. For example, the disclosed systems utilize a class-specific cascaded modulation inpainting neural network that includes cascaded modulation decoder layers to generate replacement pixels portraying a particular target object class. To illustrate, in response to user selection of a replacement region and target object class, the disclosed systems utilize a class-specific cascaded modulation inpainting neural network corresponding to the target object class to generate an inpainted digital image that portrays an instance of the target object class within the replacement region. Moreover, in one or more embodiments the disclosed systems train class-specific cascaded modulation inpainting neural networks corresponding to a variety of target object classes, such as a sky object class, a water object class, a ground object class, or a human object class.

Object class inpainting in digital images utilizing class-specific inpainting neural networks

The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object shadows for digital images utilizing corresponding geometry-aware buffer channels. For instance, in one or more embodiments, the disclosed systems generate, utilizing a height prediction neural network, an object height map for a digital object portrayed in a digital image and a background height map for a background portrayed in the digital image. The disclosed systems also generate, from the digital image, a plurality of geometry-aware buffer channels using the object height map and the background height map. Further, the disclosed systems modify the digital image to include a soft object shadow for the digital object using the plurality of geometry-aware buffer channels.

Generating soft object shadows for general shadow receivers within digital images using geometry-aware buffer channels

Systems and methods for image segmentation are described. Embodiments of the present disclosure receive an image depicting an object; generate image features for the image by performing a convolutional self-attention operation that outputs a plurality of attention-weighted values for a convolutional kernel applied at a position of a sliding window on the image; and generate label data that identifies the object based on the image features.

Transformer for efficient image segmentation

Generating and providing a panoptic inpainting interface for generating and modifying inpainted digital images

Iteratively modifying inpainted digital images based on changes to panoptic segmentation maps

Growing community of inventors

Campbell, CA, United States of America

Jianming Zhang