Adobe, Inc.

Yiwei Hu

Milos Hasan

Valentin Deschaintre

Paul Augusto Guerrero

Certain aspects and features of this disclosure relate to rendering images using target-augmented material maps. In one example, a graphics imaging application is loaded with a scene and an input material map, as well as a file for a target image. A stored, material generation prior is accessed by the graphics imaging application. This prior, as an example, is based on a pre-trained, generative adversarial network (GAN). An input material appearance from the input material map is encoded to produce a projected latent vector. The value for the projected latent vector is optimized to produce the material map that is used to render the scene, producing a material map augmented by a realistic target material appearance.

Target-augmented material maps

Embodiments are disclosed for optimizing a material graph for replicating a material of the target image. Embodiments include receiving a target image and a material graph to be optimized for replicating a material of the target image. Embodiments include identifying a non-differentiable node of the material graph, the non-differentiable node including a set of input parameters. Embodiments include selecting a differentiable proxy from a library of the selected differentiable proxy is trained to replicate an output of the identified non-differentiable node. Embodiments include generating an optimized input parameters for the identified non-differentiable node using the corresponding trained neural network and the target image. Embodiments include replacing the set of input parameters of the non-differentiable node of the material graph with the optimized input parameters. Embodiments include generating an output material by the material graph to represent the target image using the optimized input parameters for the non-differentiable node.

Growing community of inventors

New Haven, CT, United States of America

Yiwei Hu