Adobe, Inc.

Cornell University

Adove Systems Incorporated

Sylvain Philippe Paris

Elya Shechtman

Jen-Chan Jeff Chien

Vladimir Leonid Bychkovsky

Aaron Phillip Hertzmann

Eric Chan

Srinivasa Madhava Phaneendra Angara

Durga Ganesh Grandhi

Sohrab Amirghodsi

Wojciech Matusik

Robert Land Gager

Richard Zhang

Aliakbar Darabi

Frederic Durand

Chih-Yao Hsieh

Ian D Sachs

Brian Lynn Price

Gregg Darryl Wilensky

Xue Bai

Seyed Morteza Safdarnejad

Junyan Zhu

Mohit Gupta

Jacob Minyoung Huh

Sharad Baliyan

Xiaoyong Shen

Erik Andreas Härkönen

Jue Wang

Kalyan Krishna Sunkavalli

William T Freeman

Paris Smaragdis

Shmuel Avidan

Ce Liu

Simon Chen

Taesung Park

Yinglan Ma

Michael W Tao

James L McCann

Eric R Kee

Paul E Green

Kavita Bala

Ronen Barzel

Dhruv Mahajan

Robert Bailey

Ivaylo Boyadzhiev

An image processing system uses a depth-conditioned autoencoder to generate a modified image from an input image such that the modified image maintains an overall structure from the input image while modifying textural features. An encoder of the depth-conditioned autoencoder extracts a structure latent code from an input image and depth information for the input image. A generator of the depth-conditioned autoencoder generates a modified image using the structure latent code and a texture latent code. The modified image generated by the depth-conditioned autoencoder includes the structural features from the input image while incorporating textural features of the texture latent code. In some aspects, the autoencoder is depth-conditioned during training by augmenting training images with depth information. The autoencoder is trained to preserve the depth information when generating images.

Generation using depth-conditioned autoencoder

A generative neural network control system controls a generative neural network by modifying the intermediate latent space in the generative neural network. The generative neural network includes multiple layers each generating a set of activation values. An initial layer (and optionally additional layers) receives an input latent vector, and a final layer outputs an image generated based on the input latent vector. The data that is input to each layer (other than the initial layer) is referred to as data in an intermediate latent space. The data in the intermediate latent space includes activation values (e.g., generated by the previous layer or modified using various techniques) and optionally a latent vector. The generative neural network control system modifies the intermediate latent space to achieve various different effects when generating a new image.

Controlling a neural network through intermediate latent spaces

A target image is projected into a latent space of generative model by determining a latent vector by applying a gradient-free technique and a class vector by applying a gradient-based technique. An image is generated from the latent and class vectors, and a loss function is used to determine a loss between the target image and the generated image. This determining of the latent vector and the class vector, generating an image, and using the loss function is repeated until a loss condition is satisfied. In response to the loss condition being satisfied, the latent and class vectors that resulted in the loss condition being satisfied are identified as the final latent and class vectors, respectively. The final latent and class vectors are provided to the generative model and multiple weights of the generative model are adjusted to fine-tune the generative model.

Projecting images to a generative model based on gradient-free latent vector determination

This disclosure relates to methods, non-transitory computer readable media, and systems that generate a virtual long exposure image from a sequence of short exposure images portraying a moving object. In various embodiments, the image transformation system aligns two digital images in the sequence of short exposure images. The image transformation system can determine a motion vector path for the moving object between the first digital image and the second digital image. The image transformation system can also blend pixels along the motion vector path to generate a blended image representative of the motion of the moving object between the first digital image and the second digital image. The image transformation system can generate additional blended images based on consecutive pairs of images in the sequence of digital images and generates a virtual long exposure image by combining the first blended image with the additional blended images.

Generating light painting images from a sequence of short exposure images

In various example embodiments, a system and method for providing a visual example-based user interface for adjusting images is provided. In example embodiments, a new image to be adjusted is received. A plurality of basis styles is generated by applying adjustment parameters to the new image. Each of the plurality of basis styles comprises an adjusted version of the new image with an adjustment of at least one image control. A user interface is provided that positions a version of the new image in a center portion and positions the plurality of basis styles on the user interface based on the adjustment parameters applied to the new image. A control mechanism is provided over the version of the new image whereby movement of the control mechanism to a new position on the user interface causes the version of the new image to adjust accordingly.

Visual example-based user interface for adjusting photos along two dimensions

Embodiments described herein are directed to methods and systems for facilitating control of smoothness of transitions between images. In embodiments, a difference of color values of pixels between a foreground image and the background image are identified along a boundary associated with a location at which to paste the foreground image relative to the background image. Thereafter, recursive down sampling of a region of pixels within the boundary by a sampling factor is performed to produce a plurality of down sampled images having color difference indicators associated with each pixel of the down sampled images. Such color difference indicators indicate whether a difference of color value exists for the corresponding pixel. To effectuate a seamless transition, the color difference indicators are normalized in association with each recursively down sampled image.

Controlling smoothness of a transition between images

Methods and systems are provided for adjusting the brightness of images. In some implementations, an exposure bracketed set of input images produced by a camera is received. A brightness adjustment is determined for at least one input image from the set of input images. The determined brightness adjustment is applied to the input image. An output image is produced by exposure fusion from the set of input images, using the input image having the determined brightness adjustment. The output image is transmitted where, the transmitting causes display of the output image on a user device.

Preserving color in image brightness adjustment for exposure fusion

Utilizing alignment models and motion vector path blending to generate a long exposure digital image from a sequence of short exposure digital images

Environmental map generation techniques and systems are described. A digital image is scaled to achieve a target aspect ratio using a content aware scaling technique. A canvas is generated that is dimensionally larger than the scaled digital image and the scaled digital image is inserted within the canvas thereby resulting in an unfilled portion of the canvas. An initially filled canvas is then generated by filling the unfilled portion using a content aware fill technique based on the inserted digital image. A plurality of polar coordinate canvases is formed by transforming original coordinates of the canvas into polar coordinates. The unfilled portions of the polar coordinate canvases are filled using a content-aware fill technique that is initialized based on the initially filled canvas. An environmental map of the digital image is generated by combining a plurality of original coordinate canvas portions formed from the polar coordinate canvases.

Environmental map generation from a digital image

In example embodiments, systems and methods for learning and using user preferences for image adjustments are presented. In example embodiments, a new image is received. A correction parameter based on previously stored user adjustments for similar images is determined. A user style that is an adjusted version of the new image is generated by applying the correction parameter. The user style is provided on a user interface. A user adjustment is received. Based on determining that a user sample image is within a predetermined threshold of closeness to the new image, data corresponding to the user sample image is replaced with new adjustment data for the new image in a database of user sample images used to generate the correction parameter. Based on determining that no user sample images are within the predetermined threshold of closeness, new adjustment data is appended to the database used to generate the correction parameter.

Learning user preferences for photo adjustments

Growing community of inventors

Jamaica Plain, MA, United States of America

Sylvain Philippe Paris