Disney Enterprises, Inc.

Pixar

Nvidia Corporation

Eth Zurich (eidgenossische Technische Hochschule Zurich)

Eth Zurich (eidgenoessische Technische Hochschule Zurich)

Disnev Enterprises, Inc.

Jan Novak

Brian McWilliams

Mark Joseph Meyer

Thijs Vogels

Fabrice Rousselle

Alex Harvill

Wojciech Jarosz

Fabrice Pierre Armand Rousselle

Thomas Müller

Alexander Sorkine Hornung

Christopher Richard Schroers

Brent Burley

Jacopo Pantaleoni

Andrew Selle

Marios Papas

David M Adler

Ralf Habel

Wojciech Krzysztof Jarosz

Peter Kutz

Matthias Fauconneau

David Koerner

Benedikt Bitterli

Eugene d″Eon

Niko Markus Kettunen

Delio Aleardo Vicini

A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.

Temporal techniques of denoising Monte Carlo renderings using neural networks

In various examples, transmittance may be computed using a power-series expansion of an exponential integral of a density function. A term of the power-series expansion may be evaluated as a combination of values of the term for different orderings of samples in the power-series expansion. A sample may be computed from a combination of values at spaced intervals along the function and a discontinuity may be compensated for based at least on determining a version of the function that includes an alignment of a first point with a second point of the function. Rather than arbitrarily or manually selecting a pivot used to expand the power-series, the pivot may be computed as an average of values of the function. The transmittance estimation may be computed from the power-series expansion using a value used to compute the pivot (for a biased estimate) or using all different values (for an unbiased estimate).

Fractional visibility estimation using particle density for light transport simulation

Supervised machine learning using neural networks is applied to denoising images rendered by MC path tracing. Specialization of neural networks may be achieved by using a modular design that allows reusing trained components in different networks and facilitates easy debugging and incremental building of complex structures. Specialization may also be achieved by using progressive neural networks. In some embodiments, training of a neural-network based denoiser may use importance sampling, where more challenging patches or patches including areas of particular interests within a training dataset are selected with higher probabilities than others. In some other embodiments, generative adversarial networks (GANs) may be used for training a machine-learning based denoiser as an alternative to using pre-defined loss functions.

Denoising Monte Carlo renderings using progressive neural networks

According to one implementation, a system includes a computing platform having a hardware processor and a system memory storing a software code including multiple artificial neural networks (ANNs). The hardware processor executes the software code to partition a multi-dimensional input vector into a first vector data and a second vector data, and to transform the second vector data using a first piecewise-polynomial transformation parameterized by one of the ANNs, based on the first vector data, to produce a transformed second vector data. The hardware processor further executes the software code to transform the first vector data using a second piecewise-polynomial transformation parameterized by another of the ANNs, based on the transformed second vector data, to produce a transformed first vector data, and to determine a multi-dimensional output vector based on an output from the plurality of ANNs.

Piecewise-polynomial coupling layers for warp-predicting neural networks

Denoising Monte Carlo renderings using machine learning with importance sampling

Adaptive sampling in Monte Carlo renderings using error-predicting neural networks

Denoising Monte Carlo renderings using neural networks with asymmetric loss

Multi-scale architecture of denoising monte carlo renderings using neural networks

Denoising monte carlo renderings using progressive neural networks

Denoising Monte Carlo renderings using generative adversarial neural networks

According to one implementation, an image rendering system includes a computing platform having a hardware processor and a system memory storing an image denoising software code. The hardware processor executes the image denoising software code to receive an image file including multiple pixels, each pixel containing multiple depth bins, and to select a pixel including a noisy depth bin from among the pixels for denoising. The hardware processor further executes the image denoising software code to identify a plurality of reference depth bins from among the depth bins contained in one or more of the pixels, for use in denoising the noisy depth bin, and to denoise the noisy depth bin using an average of depth bin values corresponding respectively to each of the reference depth bins.

Denoising binned-depth images

A video rendering system includes a field-of-view detector, a display, and a computing platform including a hardware processor and a memory storing a multi-viewpoint video rendering software code. The hardware processor executes the multi-viewpoint video rendering software code to parameterize visible surfaces in a scene to define multiple texels for each visible surface, precompute one or more illumination value(s) for each texel of each visible surface, and for each texel of each visible surface, store the illumination value(s) in a cache assigned to the texel. In addition, the multi-viewpoint video rendering software code receives a perspective data from the field-of-view detector identifying one of multiple permissible perspectives for viewing the scene, and renders the scene on the display in real-time with respect to receiving the perspective data, based on the identified perspective and using one or more of the illumination value(s) precomputed for each texel of each visible surface.

Augmented video rendering

The disclosure provides an approach for rendering heterogeneous polydisperse granular media. In one aspect, a rendering application renders such granular media using a combination of explicit path tracing and accelerated path construction using proxy path tracing, shell tracing, and volumetric path tracing. In proxy path tracing in particular, the rendering application instantiates proxy geometry in the form of a bounding sphere and determines internal scattering in the grain using a precomputed grain scattering distribution function that relates incident and outgoing radiance functions on the bounding sphere. In shell tracing, the rendering application uses shells to aggregate many grain interactions into a single step. The rendering application derives a continuous liquid/volume equivalent to the granular material based on the material's optical properties and selects a precomputed shell transport function (STF) to use from a database by interpolating nearby database entries using radiative transfer equation (RTE) parameters at the shell's center.

Efficient rendering of heterogeneous polydisperse granular media

Growing community of inventors

Meilen, Switzerland

Jan Novak