Disney Enterprises, Inc.

Eth Zurich (eidgenossische Technische Hochschule Zurich)

Eth Zurich (eidgenoessische Technische Hochschule Zurich)

Disney Enterprise, Inc.

Marios Papas

Wojciech Jarosz

Wojciech Matusik

Christopher Richard Schroers

Szymon Marek Rusinkiewicz

Tim A Weyrich

Steve Marschner

Farnood Salehi

Wenzel A Jakob

Marco Manzi

Xianyao Zhang

Mark Joseph Meyer

Philip John Jackson

Tunc Ozan Aydin

Abdelaziz Djelouah

Bernd Bickel

Jan Novak

Derek Nowrouzezahrai

Thijs Vogels

Thomas Müller

David M Adler

Ralf Habel

Wojciech Krzysztof Jarosz

Christian Regg

Ian Mcgonigal

Gerhard Roethlin

Karlis Martins Briedis

Shilin Zhu

Markus Plack

Mark A Meyer

Johannes Meng

Carsten Dachsbacher

Thomas Houit

Henrik D Dahlberg

Gerhard Röthlin

Andre C Mazzone

Per H Christensen

Melvin Ott

A system includes a hardware processor and a system memory storing software code and one or more machine learning (ML) models. The hardware processor is configured to execute the software code to train a first ML model of the one or more ML models as a denoising feature selector, generate, using the trained first ML model a plurality of candidate feature sets, and identify a best volumetric feature set of the plurality of candidate feature sets using a predetermined selection criterion. The hardware processor is further configured to execute the software code to train, using the identified best volumetric feature set, one of the first ML model or a second ML model of the one or more ML models as a denoiser, receive an image including noise due to rendering, and denoise, using the trained denoiser, the noise due to rendering to produce a denoised image.

Volume denoising with feature selection

Certain aspects of the present disclosure provide techniques for adaptive sampling for rendering using deep learning. This includes receiving, at a sampler in a rendering pipeline, a plurality of rendered pixel data, wherein the sampler includes a first machine learning (ML) model. It further includes generating a sampling map for the rendering pipeline using the first ML model and the plurality of rendered pixel data, including predicting a plurality of pixel values in the sampling map based on a generated distribution of pixel values. It further includes rendering an image using the sampler, the sampling map, and a denoiser in the rendering pipeline.

Adaptive sampling using deep learning

A system includes a pre-processor configured to receive three-dimensional (3-D) image data, flatten the 3-D image data to produce corresponding two-dimensional (2-D) image data, and concatenate the 3-D image data and the corresponding 2-D image data to provide concatenated image data. The system further includes an encoder including one or more first neural networks (NNs), the encoder configured to use the one or more first NNs to encode the concatenated image data to provide encoded data, a decoder including one or more second NNs, the decoder configured to use the one or more second NNs to decode the encoded data to provide decoded data, and a reconstructor including a plurality of hybrid 2-D/3-D reconstructors configured to reconstruct the decoded data to provide a denoised 3-D output image corresponding to the 3-D image data.

Hybrid two-dimensional and three-dimensional denoiser

One embodiment of the present invention sets forth a technique for performing frame interpolation. The technique includes generating (i) a first set of feature maps based on a first set of rendering features associated with a first key frame, (ii) a second set of feature maps based on a second set of rendering features associated with a second key frame, and (iii) a third set of feature maps based on a third set of rendering features associated with a target frame. The technique also includes applying one or more neural networks to the first, second, and third set of feature maps to generate a set of mappings from a first set of pixels in the first key frame to a second set of pixels in the target frame. The technique further includes generating the target frame based on the set of mappings.

Frame interpolation for rendered content

Techniques are described for designing and manufacturing a surface that produces a desired image when illuminated by a light source. As described, the desired image may be decomposed into a collection of Gaussian kernels (referred to as Gaussians). A shape of a micropatch lens corresponding to each Gaussian may be determined, and the resulting micropatch lenses may be assembled to form a highly continuous surface that will cast an approximation of the desired image formed form the sum of a plurality of Gaussian caustics. The disclosed techniques may be used to create a design for a light-redirecting surface amenable to milling (or other manufacturing process).

Reflective and refractive surfaces configured to project desired caustic pattern

The disclosure provides an approach for rendering heterogeneous polydisperse granular media. In one aspect, a rendering application renders such granular media using a combination of explicit path tracing and accelerated path construction using proxy path tracing, shell tracing, and volumetric path tracing. In proxy path tracing in particular, the rendering application instantiates proxy geometry in the form of a bounding sphere and determines internal scattering in the grain using a precomputed grain scattering distribution function that relates incident and outgoing radiance functions on the bounding sphere. In shell tracing, the rendering application uses shells to aggregate many grain interactions into a single step. The rendering application derives a continuous liquid/volume equivalent to the granular material based on the material's optical properties and selects a precomputed shell transport function (STF) to use from a database by interpolating nearby database entries using radiative transfer equation (RTE) parameters at the shell's center.

Efficient rendering of heterogeneous polydisperse granular media

A method for creating a replication material corresponding to the appearance of a translucent or partially translucent target material. The appearance of the target material can be measured or may be prescribed by a user. The method includes receiving by a processor optical data related to a target subsurface scattering parameter of the target material. Once the processor has received the optical or light characteristic data, the method includes determining by the processor a replication pigment concentration to replicate the appearance of the target material caused by the target subsurface scattering parameter. The processor determines this concentration based on a plurality of pigment subsurface scattering parameters corresponding to a plurality of stored pigment concentrations in the computing device. Once the replication pigment concentration has been determined, the method includes creating, physically or virtually, the replication material by combining the pigment concentration with a base material.

Method of fabricating translucent materials with desired appearance

The disclosure provides an approach for rendering granular media. According to one aspect of the disclosure, granular media are rendered using bidirectional point scattering distribution functions (BPSDFs). The dimensionality of BPSDFs may be reduced by making certain assumptions, such as random orientations of grains, thereby simplifying light transport for computational efficiency. To generate a BPSDF from a grain, light transport may be precomputed using a Monte Carlo simulation in which photons are shot onto the grain from all directions. The precomputed BPSDF may be used, during rendering, for describing the interactions within grains. When a light ray traced during rendering intersects proxy geometry which replaces grain geometry, the BPSDF may be evaluated to determine light transport. By repeating this process for many light rays in a Monte Carlo simulation, the light propagation through the granular medium may be determined.

Bidirectional point distribution functions for rendering granular media

Techniques are described for designing and manufacturing a refractive surface that produces a desired image when placed over a target image. The refractive lens surface may include a set of lens patches, each of which indexes a region on the source image to refract light from the indexed region to recreate a patch of the target image. And together, the lenses reproduce the target image. In one embodiment, the refractive geometry of the lens surface (i.e., the shape of each lens) is determined by formulating and efficiently determining a solution to an inverse light transport problem. The solution may account for additional constraints imposed by the physical manufacturing procedure. Doing so results in a design for a refractive surface amenable to milling (or other manufacturing process).

Refractive steganography lens and method for determining milling plan for same

Growing community of inventors

Zurich, Switzerland

Marios Papas