Adobe, Inc.

Julien Philip

Jianming Zhang

Kalyan Krishna Sunkavalli

Xin Sun

He Zhang

Zhixin Shu

Zexiang Xu

Sai Bi

Valentin Deschaintre

Qiangeng Xu

David Nicholson Griffiths

Yichen Sheng

The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object shadows for digital images utilizing corresponding geometry-aware buffer channels. For instance, in one or more embodiments, the disclosed systems generate, utilizing a height prediction neural network, an object height map for a digital object portrayed in a digital image and a background height map for a background portrayed in the digital image. The disclosed systems also generate, from the digital image, a plurality of geometry-aware buffer channels using the object height map and the background height map. Further, the disclosed systems modify the digital image to include a soft object shadow for the digital object using the plurality of geometry-aware buffer channels.

Generating soft object shadows for general shadow receivers within digital images using geometry-aware buffer channels

Methods and systems disclosed herein relate generally to radiance field gradient scaling for unbiased near-camera training. In a method, a processing device accesses an input image of a three-dimensional environment comprising a plurality of pixels, each pixel comprising a pixel color. The processing device determines a camera location based on the input image and a ray from the camera location in a direction of a pixel. The processing device integrates sampled information from a volumetric representation along the ray from the camera location to obtain an integrated color. The processing device trains a machine learning model configured to predict a density and a color, comprising minimizing a loss function using a scaling factor that is determined based on a distance between the camera location and a point along the ray. The processing device outputs the trained ML model for use in rendering an output image.

Radiance field gradient scaling for unbiased near-camera training

A scene modeling system receives a plurality of input two-dimensional (2D) images corresponding to a plurality of views of an object and a request to display a three-dimensional (3D) scene that includes the object. The scene modeling system generates an output 2D image for a view of the 3D scene by applying a scene representation model to the input 2D images. The scene representation model includes a point cloud generation model configured to generate, based on the input 2D images, a neural point cloud representing the 3D scene. The scene representation model includes a neural point volume rendering model configured to determine, for each pixel of the output image and using the neural point cloud and a volume rendering process, a color value. The scene modeling system transmits, responsive to the request, the output 2D image. Each pixel of the output image includes the respective determined color value.

Point-based neural radiance field for three dimensional scene representation

Directional propagation editing techniques are described, in one example, a digital image, a depth map, and a direction are obtained by an image editing system. The image editing system then generates features. To do so, the image editing system generates features from the digital image and the depth map for each pixel based on the direction, e.g., until an edge of the digital image is reached. In an implementation, instead of storing a value of the depth directly, a ratio is stored based on a depth in the depth map and a depth of a point along the direction. The image editing system then forms a feature volume using the features, e.g., as three dimensionally stacked features. The feature volume is employed by the image editing system as part of editing the digital image to form an edited digital image.

Directional editing of digital images

Growing community of inventors

London, United Kingdom

Julien Philip