Adobe, Inc.

Ionut Mironica

Oliver Brdiczka

Vlad-Constantin Lungu-Stan

Yijun Li

Alexandru Vasile Costin

Oscar Bolaños

Andreea Birhalâ

Richard Zhang

Adrian-Stefan Ungureanu

George Bogdan Avram

Stefan Daniel Dumitrescu

Marian Lupascu

The present disclosure relates to systems, methods, and non-transitory computer readable media for generating digital designs utilizing a diffusion neural network to preserve readability and design composition while modifying image content background images and design assets. In some embodiments, the disclosed systems access a text prompt defining visual attributes of a digital design. Furthermore, the disclosed systems generate a modified text prompt by replacing chromatic information within the text prompt. Additionally, the disclosed systems determine an adaptive strength for a diffusion neural network from the text prompt. Also, the disclosed systems generate a modified digital design utilizing the diffusion neural network to process the modified text prompt according to the adaptive strength.

Generating graphic designs by exploiting contrast through generative editing

Systems and methods for generating full designs from text include retrieving a plurality of document templates based on a design prompt, and filtering the document templates based on an image prompt. A document template is then selected based on the filtering, and a document is generated based on the document template and the image prompt. Embodiments are further configured to generate content from one or more of the prompts, where the content is included in the final design document.

Generating full designs from text

The present disclosure relates to systems, non-transitory computer-readable media, and methods for utilizing a design language model and a generative language model to generate digital design documents with design variations. In particular embodiments, the disclosed systems implement the design language model to tokenize the design of a document into a sequence of language tokens. For example, the disclosed systems tokenize visual elements and a layout of the document—in addition to optional user-added content. The generative language model utilizes the sequence of language tokens to predict a next language token representing a suggested design variation. Based on the predicted language token, the disclosed systems generate a modified digital design document visually portraying the suggested design variation. Further, in one or more embodiments, the disclosed systems perform iterative refinements to the modified digital design document.

Utilizing a transformer-based generative language model to generate digital design document variations

The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately, efficiently, and flexibly restoring degraded digital images utilizing a deep learning framework for repairing local defects, correcting global imperfections, and/or enhancing depicted faces. In particular, the disclosed systems can utilize a defect detection neural network to generate a segmentation map indicating locations of local defects within a digital image. In addition, the disclosed systems can utilize an inpainting algorithm to determine pixels for inpainting the local defects to reduce their appearance. In some embodiments, the disclosed systems utilize a global correction neural network to determine and repair global imperfections. Further, the disclosed systems can enhance one or more faces depicted within a digital image utilizing a face enhancement neural network as well.

Restoring degraded digital images through a deep learning framework

This disclosure describes methods, non-transitory computer readable storage media, and systems that utilize one or more stages of a two-stage image colorization neural network to colorize or re-colorize digital images. In one or more embodiments, the disclosed system generates a color digital image from a grayscale digital image by utilizing a colorization neural network. Additionally, the disclosed system receives one or more inputs indicating local hints comprising one or more color selections to apply to one or more objects of the color digital image. The disclosed system then utilizes a re-colorization neural network to generate a modified digital image from the color digital image by modifying one or more colors of the object(s) based on the luminance channel, color channels, and selected color(s).

Generating colorized digital images utilizing a re-colorization neural network with local hints

The present disclosure relates to an image artifact removal system that improves digital images by removing complex artifacts caused by image compression. For example, in various implementations, the image artifact removal system builds a generative adversarial network that includes a generator neural network and a discriminator neural network. In addition, the image artifact removal system trains the generator neural network to reduce and eliminate compression artifacts from the image by synthesizing or retouching the compressed digital image. Further, in various implementations, the image artifact removal system utilizes dilated attention residual layers in the generator neural network to accurately remove compression artifacts from digital images of different sizes and/or having different compression ratios.

Removing compression artifacts from digital images and videos utilizing generative machine-learning models

Techniques for generating style-transferred images are provided. In some embodiments, a content image, a style image, and a user input indicating one or more modifications that operate on style-transferred images are received. In some embodiments, an initial style-transferred image is generated using a machine learning model. In some examples, the initial style-transferred image comprises features associated with the style image applied to content included in the content image. In some embodiments, a modified style-transferred image is generated by modifying the initial style-transferred image based at least in part on the user input indicating the one or more modifications.

Universal style transfer using multi-scale feature transform and user controls

In implementations of correcting dust and scratch artifacts in digital images, an artifact correction system receives a digital image that depicts a scene and includes a dust or scratch artifact. The artifact correction system generates, with a generator of a generative adversarial neural network (GAN), a feature map from the digital image that represents features of the dust or scratch artifact and features of the scene. A training system can train the generator adversarially to reduce visibility of dust and scratch artifacts in digital images against a discriminator, and train the discriminator to distinguish between reconstructed digital images generated by the generator and real-world digital images. The artifact correction system generates, from the feature map and with the generator, a reconstructed digital image that depicts the scene of the digital image and reduces visibility of the dust or scratch artifact of the digital image.

Correcting dust and scratch artifacts in digital images

The present disclosure relates to systems, non-transitory computer-readable media, and methods for utilizing a design language model and a generative language model to generate digital design documents with design variations. In particular embodiments, the disclosed systems implement the design language model to tokenize the design of a document into a sequence of language tokens. For example, the disclosed systems tokenize visual elements and a layout of the document&#x2014;in addition to optional user-added content. The generative language model utilizes the sequence of language tokens to predict a next language token representing a suggested design variation. Based on the predicted language token, the disclosed systems generate a modified digital design document visually portraying the suggested design variation. Further, in one or more embodiments, the disclosed systems perform iterative refinements to the modified digital design document.

This disclosure involves automatically generating a typographical image using an image and a text document. Aspects of the present disclosure include detecting a region of interest from the image and generating an object template from the detected region of interest. The object template defines the areas of the image, in which words of the text document are inserted. A text rendering protocol is executed to iteratively insert the words of the text document into the available locations of the object template. The typographical image is generated by rendering each word of the text document onto the available location assigned to the word.

Systems and methods for generating typographical images or videos

Systems and methods are disclosed for automatically correcting a rotation of an image. One method comprises receiving an input image to be processed for rotation correction, and providing the input image to a machine learning model configured to predict a rotation angle and an orientation of an image. The machine learning model may comprise a convolutional neural network utilizing one or more rectangle-shaped filters to extract features from the input image. A rotation angle may be determined using a first branch of fully connected layer configured for a regression task of the machine learning model, and an orientation of the image may be determined using a second branch of fully connected layer configured for a classification task of the machine learning model. The input image may then be rotated based on the determined rotation angle and orientation.

Growing community of inventors

Bucharest, Romania

Ionut Mironica