University of California

Adobe, Inc.

Fotonation Limited

Ravi Ramamoorthi

Zexiang Xu

Kartik Venkataraman

Kalyan Krishna Sunkavalli

David J Kriegman

Manohar B Srikanth

Priyam Chatterjee

Milos Hasan

Sai Bi

Alexandr Kuznetsov

Krishna Bhargava Mullia Lakshminarayana

Yannick Hold-Geoffroy

Fujun Luan

Nima Khademi Kalantari

Xuezheng Wang

Miloš Hašan

A scene modeling system accesses a three-dimensional (3D) scene including a 3D object. The scene modeling system applies a silhouette bidirectional texture function (SBTF) model to the 3D object to generate an output image of a textured material rendered as a surface of the 3D object. Applying the SBTF model includes determining a bounding geometry for the surface of the 3D object. Applying the SBTF model includes determining, for each pixel of the output image, a pixel value based on the bounding geometry. The scene modeling system displays, via a user interface, the output image based on the determined pixel values.

Neural networks to render textured materials on curved surfaces

A scene reconstruction system renders images of a scene with high-quality geometry and appearance and supports view synthesis, relighting, and scene editing. Given a set of input images of a scene, the scene reconstruction system trains a network to learn a volume representation of the scene that includes separate geometry and reflectance parameters. Using the volume representation, the scene reconstruction system can render images of the scene under arbitrary viewing (view synthesis) and lighting (relighting) locations. Additionally, the scene reconstruction system can render images that change the reflectance of objects in the scene (scene editing).

Scene reconstruction using geometry and reflectance volume representation of scene

Methods and systems disclosed herein relate generally to surface-rendering neural networks to represent and render a variety of material appearances (e.g., textured surfaces) at different scales. The system includes receiving image metadata for a texel that includes position, incoming and outgoing radiance direction, and a kernel size. The system applies a offset-prediction neural network to the query to identify an offset coordinate for the texel. The system inputs the offset coordinate to a data structure to determine a feature vector for the texel of the textured surface. The reflectance feature vector is then processed using a decoder neural network to estimate a light-reflectance value of the texel, at which the light-reflectance value is used to render the texel of the textured surface.

Rendering textured surface using surface-rendering neural networks

Enhanced methods and systems for generating both a geometry model and an optical-reflectance model (an object reconstruction model) for a physical object, based on a sparse set of images of the object under a sparse set of viewpoints. The geometry model is a mesh model that includes a set of vertices representing the object's surface. The reflectance model is SVBRDF that is parameterized via multiple channels (e.g., diffuse albedo, surface-roughness, specular albedo, and surface-normals). For each vertex of the geometry model, the reflectance model includes a value for each of the multiple channels. The object reconstruction model is employed to render graphical representations of a virtualized object (a VO based on the physical object) within a computation-based (e.g., a virtual or immersive) environment. Via the reconstruction model, the VO may be rendered from arbitrary viewpoints and under arbitrary lighting conditions.

Generating enhanced three-dimensional object reconstruction models from sparse set of object images

Devices, systems and methods for generating high dynamic range images and video from a set of low dynamic range images and video using convolution neural networks (CNNs) are described. One exemplary method for generating high dynamic range visual media includes generating, using a first CNN to merge a first set of images having a first dynamic range, a final image having a second dynamic range that is greater than the first dynamic range. Another exemplary method for generating training data includes generating sets of static and dynamic images having a first dynamic range, generating, based on a weighted sum of the set of static images, a set of ground truth images having a second dynamic range greater than the first dynamic range, and replacing at least one of the set of dynamic images with an image from the set of static images to generate a set of training images.

Generation of high dynamic range visual media

Systems and methods in accordance with embodiments of this invention perform depth regularization and semiautomatic interactive matting using images. In an embodiment of the invention, the image processing pipeline application directs a processor to receive (i) an image (ii) an initial depth map corresponding to the depths of pixels within the image, regularize the initial depth map into a dense depth map using depth values of known pixels to compute depth values of unknown pixels, determine an object of interest to be extracted from the image, generate an initial trimap using the dense depth map and the object of interest to be extracted from the image, and apply color image matting to unknown regions of the initial trimap to generate a matte for image matting.

System and methods for depth regularization and semiautomatic interactive matting using RGB-D images

Growing community of inventors

Carlsbad, CA, United States of America

Ravi Ramamoorthi