Adobe, Inc.

Byungmoon Kim

Zhili Chen

Daichi Ito

Gahye Park

Nathan Aaron Carr

Joon-Young Lee

Jinwoong Jung

Zhe Lin

Jonathan Worthen Brandt

Sunil Sharadchandra Hadap

Gavin Miller

Jinyi Kwon

Yuan Gao

Biao Jia

Jihyun Lee

Jingwan Lu

Eunyee Koh

Stephen J DiVerdi

Gavin Stuart Peter Miller

Chen Fang

Yuyan Song

Gagan Singhal

Duygu Ceylan

Grayson Squier Lang

Brian David Williams

Henricus Cabanier

Cory Lynn Edwards

Yeojin Kim

Jose Ignacio Echevarria Vallespi

JaeGwang Lim

Jun Xing

Radomir Mech

Brian Lynn Price

Gregg Darryl Wilensky

Linjie Luo

Jimei Yang

Ning Xu

Duygu Ceylan Aksit

Li-Yi Wei

Vladimir Kim

Gavin S P Miller

Walter Wei-Tuh Chang

Niloy Jyoti Mitra

Jonathan Eisenmann

Ondrej Stava

Srinivasa Madhava Phaneendra Angara

Qi Sun

Liwen Hu

Aron Monszpart

Antoine Amanieux

Laurel Alisa Warrell

Min Jung Son

Some embodiments involve a reinforcement learning based framework for training a natural media agent to learn a rendering policy without human supervision or labeled datasets. The reinforcement learning based framework feeds the natural media agent a training dataset to implicitly learn the rendering policy by exploring a canvas and minimizing a loss function. Once trained, the natural media agent can be applied to any reference image to generate a series (or sequence) of continuous-valued primitive graphic actions, e.g., sequence of painting strokes, that when rendered by a synthetic rendering environment on a canvas, reproduce an identical or transformed version of the reference image subject to limitations of an action space and the learned rendering policy.

Reinforcement learning-based techniques for training a natural media agent

Systems, methods, and non-transitory computer-readable media are disclosed for modifying voxel-based 3D representations using 3D digital brush tools and/or resolution filters. For example, the disclosed systems can utilize 3D digital brush tools (e.g., a digital blur brush tool, a digital smudge brush tool, and/or a digital melt brush tool) to identify and modify one or more voxels within a 3D representation using multiple buffers of visual properties. Additionally, the disclosed systems can modify one or more voxels within a 3D representation by rendering the one or more voxels at varying levels of detail using an octree (e.g., a mosaic filter tool). In particular, the disclosed systems can identify one or more voxels within an octree that are smaller than a target voxel size. Moreover, the disclosed systems can combine the identified one or more voxels within the octree to render the 3D representation at varying levels of detail.

Modifying voxel resolutions within three-dimensional representations

Certain embodiments involve a graphics manipulation application using brushstroke parameters that include a maximum alpha-deposition parameter and a fractional alpha-deposition parameter. For instance, the graphics manipulation application uses an alpha flow increment computed from the maximum alpha-deposition parameter and the fractional alpha-deposition parameter to compute an output canvas color. In some embodiments, if the current canvas opacity exceeds or equals the maximum alpha-deposition parameter, the current canvas opacity is selected as the output canvas opacity. Otherwise, the graphics manipulation application computes the output canvas opacity by increasing the current canvas opacity based on the alpha flow increment. The graphics manipulation application updates a canvas portion affected by a brushstroke input to include the output canvas opacity and the output canvas color.

Digital overpainting controlled by opacity and flow parameters

Modifying three-dimensional representations using digital brush tools

A non-photorealistic image rendering system and related techniques are described herein that train and implement machine learning models to reproduce digital images in accordance with various painting styles and constraints. The image rendering system can include a machine learning system that utilizes actor-critic based reinforcement learning techniques to train painting agents (e.g., models that include one or more neural networks) how to transform images into various artistic styles with minimal loss between the original images and the transformed images. The image rendering system can generate constrained painting agents, which correspond to painting agents that are further trained to reproduce images in accordance with one or more constraints. The constraints may include limitations of the color, width, size, and/or position of brushstrokes within reproduced images. These constrained painting agents may provide users with robust, flexible, and customizable non-photorealistic painting systems.

Learned model-based image rendering

Stroke operation prediction techniques and systems for three-dimensional digital content are described. In one example, stroke operation data is received that describes a stroke operation input via a user interface as part of the three-dimensional digital content. A cycle is generated that defines a closed path within the three-dimensional digital content based on the input stroke operation and at least one other stroke operation in the user interface. A surface is constructed based on the generated cycle. A predicted stroke operation is generated based at least in part on the constructed surface. The predicted stroke operation is then output in real time in the user interface as part of the three-dimensional digital content as the stroke operation data is received.

Stroke operation prediction for three-dimensional digital content

Embodiments herein describe a framework for classifying images. In some embodiments, it is determined whether an image includes synthetic image content. If it does, characteristics of the image are analyzed to determine if the image includes characteristics particular to panoramic images (e.g., possess a threshold equivalency of pixel values among the top and/or bottom boundaries of the image, or a difference between summed pixel values of the pixels comprising the right vertical boundary of the image and summed pixel values of the pixels comprising the left vertical boundary of the image being less than or equal to a threshold value). If the image includes characteristics particular to panoramic images, the image is classified as a synthetic panoramic image. If the image is determined to not include synthetic image content, a neural network is applied to the image and the image is classified as one of non-synthetic panoramic or non-synthetic non-panoramic.

Classifying panoramic images

In some embodiments, a graphics manipulation application accesses, for a received brushstroke input, brushstroke parameters that include a maximum alpha-deposition parameter and a fractional alpha-deposition parameter. The graphics manipulation application computes an alpha flow increment from the maximum alpha-deposition parameter and the fractional alpha-deposition parameter. The graphics manipulation application computes an output canvas color from the alpha flow increment and a current canvas opacity, and obtains an output canvas opacity based on the current canvas opacity and the maximum alpha-deposition parameter. If the current canvas opacity exceeds or equals the maximum alpha-deposition parameter, the current canvas opacity is selected as the output canvas opacity. Otherwise, the graphics manipulation application computes the output canvas opacity by increasing the current canvas opacity based on the alpha flow increment. The graphics manipulation application updates a canvas portion affected by the brushstroke input to include the output canvas opacity and the output canvas color.

Techniques and systems are described herein that support improved object painting in digital images through use of perspectives and transfers in a digital medium environment. In one example, a user interacts with a two-dimensional digital image in a user interface output by a computing device to apply digital paint. The computing device fits a three-dimensional model to an object within the image, e.g., the face. The object, as fit to the three-dimensional model, is used to support output of a plurality of perspectives of a view of the object with which a user may interact to digitally paint the object. As part of this, digital paint as specified through the user inputs is applied directly by the computing device to a two-dimensional texture map of the object. This may support transfer of digital paint by a computing device between objects by transferring the digital paint using respective two-dimensional texture maps.

Object painting through use of perspectives or transfers in a digital medium environment

Combined selection tool techniques are described in which selection of portions within an image is enabled via a tool configured to selectively switch between a coarse selection mode and a refinement selection mode. In one or more implementations, an image is exposed for editing in a user interface and input is obtained to select portions of the image using the tool. The selection mode is automatically toggled between the coarse selection mode and refinement selection mode based on characteristics of the input, such as position and velocity of the input or gestures mapped to the input. Then, selection boundaries defining the selection of portions of the image are modified in accordance with the input. In one approach, the coarse selection mode corresponds to a graph cut mechanism and the refinement selection mode corresponds to a level set mechanism.

Combined selection tool

Methods and systems for reconstructing surfaces of an object using regional level sets (RLS) are disclosed. A scanning system scans an object and generates a point cloud. An RLS is iteratively determined as solution to a differential equation constrained by the point cloud. The RLS is a 2-tuple, where the first component corresponds to a region identification and the second component corresponds to the solution of the differential equation. The space around the point cloud is iteratively segmented into a plurality of regions. A single solution to the differential equation is applied, encompassing all the regions. The solution in regions of the space corresponding to the finer structures within the point cloud are modeled similar to the coarser regions. The solution in a particular region is iteratively based on the solution in the neighboring regions. An RLS is enabled to reconstruct thinner or smaller structures or surfaces of the object.

Reconstruction of object surfaces from a point cloud by employing regional level sets

The present disclosure includes methods and systems for modifying orientation of a spherical panorama digital image based on an inertial measurement device. In particular, one or more embodiments of the disclosed systems and methods correct for tilt and/or roll in a digital camera utilized to capture a spherical panorama digital images by detecting changes in orientation to an inertial measurement device and generating an enhanced spherical panorama digital image based on the detect changes. In particular, in one or more embodiments, the disclosed systems and methods modify orientation of a spherical panorama digital image in three-dimensional space based on changes in orientation to an inertial measurement device and resample pixels based on the modified orientation to generate an enhanced spherical panorama digital image.

Utilizing an inertial measurement device to adjust orientation of panorama digital images

Growing community of inventors

Sunnyvale, CA, United States of America

Byungmoon Kim