Adobe, Inc.

Netapp, Inc.

Jing Shi

Ning Xu

Franck Dernoncourt

Trung Huu Bui

Zhe Lin

Brian M Hackworth

Brian J Pawlowski

Wei Xiong

Hyun Joon Jung

Mrinmoy Majumdar

Daniel Nghia Truong

One or more aspects of a method, apparatus, and non-transitory computer readable medium include obtaining an input description and an input image depicting a subject, encoding the input description using a text encoder of an image generation model to obtain a text embedding, and encoding the input image using a subject encoder of the image generation model to obtain a subject embedding. A guidance embedding is generated by combining the subject embedding and the text embedding, and then an output image is generated based on the guidance embedding using a diffusion model of the image generation model. The output image depicts aspects of the subject and the input description.

Personalized text-to-image generation

This disclosure describes methods, non-transitory computer readable storage media, and systems that utilize weakly supervised graph matching to align an ungrounded label graph and a visual graph corresponding to a digital image. Specifically, the disclosed system utilizes a label embedding model to generate label graph embeddings from the ungrounded label graph and a visual embedding network to generate visual graph embeddings from the visual graph. Additionally, the disclosed system determines similarity metrics indicating the similarity of pairs of label graph embeddings and visual graph embeddings. The disclosed system then generates a semantic scene graph by utilizing a graph matching algorithm to align the ungrounded label graph and the visual graph based on the similarity metrics. In some embodiments, the disclosed system utilizes contrastive learning to modify the embedding models. Furthermore, in additional embodiments, the disclosed system utilizes the semantic scene graph to train a scene graph generation neural network.

Generating semantic scene graphs from ungrounded label graphs and visual graphs for digital images

This disclosure involves executing artificial intelligence models that infer image editing operations from natural language requests spoken by a user. Further, this disclosure performs the inferred image editing operations using inferred parameters for the image editing operations. Systems and methods may be provided that infer one or more image editing operations from a natural language request associated with a source image, locate areas of the source that are relevant to the one or more image editing operations to generate image masks, and performing the one or more image editing operations to generate a modified source image.

Artificial intelligence techniques for performing image editing operations inferred from natural language requests

The present disclosure relates to systems, methods, and non-transitory computer-readable media that utilize a neural network having a long short-term memory encoder-decoder architecture to progressively modify a digital image in accordance with a natural language request. For example, in one or more embodiments, the disclosed systems utilize a language-to-operation decoding cell of a language-to-operation neural network to sequentially determine one or more image-modification operations to perform to modify a digital image in accordance with a natural language request. In some cases, the decoding cell determines an image-modification operation to perform partly based on the previously used image-modification operations. The disclosed systems further utilize the decoding cell to determine one or more operation parameters for each selected image-modification operation. The disclosed systems utilize the image-modification operation(s) and operation parameter(s) to modify the digital image (e.g., by generating one or more modified digital images) via the decoding cell.

Performing global image editing using editing operations determined from natural language requests

A system management tool provides a comprehensive summary of operational information of a storage system. For example, the system management tool presents a digital dashboard ('dashboard') in a graphical user interface. The system management tool also presents, via the dashboard, views that correspond to different dimensions of the storage system. In response to expansion of a selected view, the system management tool can automatically collapse the other views into a non-expanded views or move the other views to visible areas remaining after expansion of the selected view. Thus, all of the views remain continuously visible via the dashboard despite the modification to the appearance of the selected view.

Growing community of inventors

Rochester, NY, United States of America

Jing Shi