Disney Enterprises, Inc.

Eidgenossische Technische Hochschule Zurich

Eth Zurich

Olga Sorkine-Hornung

Alexander Sorkine-Hornung

Changil Kim

Christopher Richard Schroers

Kaan Yücer

Henning Zimmer

Mario Botsch

Yifan Wang

Katja Wolff

Abdelaziz Djelouah

Ahmet Cengiz Öztireli

Victor Cornillere

Shihao Wu

Kenshi Takayama

Daniele Panozzo

Alec Stefan Jacobson

A scaling application estimates a downscaling kernel used to generate a downscaled image. The scaling application upscales the downscaled image based on the estimated downscaling kernel, thereby generating a higher resolution version of the downscaled image with minimal visual artifacts. The scaling application includes various networks that perform the above operations. A kernel mapping network generates a degradation map based on the estimated downscaling kernel. A degradation-aware generator network generates a reconstructed image based on the downscaled image and the degradation map. A kernel discriminator network generates an image delta that reflects visual artifacts present in the reconstructed image. The scaling application includes a parameter optimizer that iteratively modifies the estimated downscaling kernel to reduce visual artifacts indicated in the image delta. Via one or more iterations, the scaling application generates a reasonably accurate estimation of the downscaling kernel and can then upscale the downscaled image with reduced visual artifacts.

Techniques for upscaling images generated with undetermined downscaling kernels

In various embodiments, a differentiable rendering application enables an inverse rendering application to infer attributes associated with a 3D scene. In operation, the differentiable rendering application renders an image based on a first set of points associated with the 3D scene. The differentiable rendering application then generates an artificial gradient that approximates a change in a value of a first pixel included in the image with respect to a change in an attribute of a first point included in the first set of points. Subsequently, the inverse rendering application performs optimization operation(s) on the first point based on the artificial gradient to generate a second set of points. Notably, an error associated with the second set of points is less than an error associated with the first set of points.

Techniques for performing point-based inverse rendering

The present disclosure relates to techniques for reconstructing an object in three dimensions that is captured in a set of two-dimensional images. The object is reconstructed in three dimensions by computing depth values for edges of the object in the set of two-dimensional images. The set of two-dimensional images may be samples of a light field surrounding the object. The depth values may be computed by exploiting local gradient information in the set of two-dimensional images. After computing the depth values for the edges, depth values between the edges may be determined by identifying types of the edges (e.g., a texture edge, a silhouette edge, or other type of edge). Then, the depth values from the set of two-dimensional images may be aggregated in a three-dimensional space using a voting scheme, allowing the reconstruction of the object in three dimensions.

Object reconstruction from dense light fields via depth from gradients

Enhanced removing of noise and outliers from one or more point sets generated by image-based 3D reconstruction techniques is provided. In accordance with the disclosure, input images and corresponding depth maps can be used to remove pixels that are geometrically and/or photometrically inconsistent with the colored surface implied by the input images. This allows standard surface reconstruction methods (such as Poisson surface reconstruction) to perform less smoothing and thus achieve higher quality surfaces with more features. In some implementations, the enhanced point-cloud noise removal in accordance with the disclosure can include computing per-view depth maps, and detecting and removing noisy points and outliers from each per-view point cloud by checking if points are consistent with the surface implied by the other input views.

Point cloud noise and outlier removal for image-based 3D reconstruction

System, method, and computer program product to perform an operation, comprising sampling a plurality of points and a plurality of segments of a curve on a surface of a three-dimensional model, storing each sampled point as a respective vertex of a plurality of vertices and each sampled segment as a respective half-edge in a curve network of the model surface, upon determining that a first half-edge and a second half-edge connect two of the plurality of vertices, generating a first halfchain connecting the first half-edge and the second half-edge, wherein each connected vertex comprises either a corner or an open endpoint, and upon determining that three consecutive halfchains form a loop comprising at least three corners, generating a first patch for a space enclosed by the loop, wherein the first patch is represented as a quad mesh with a respective set of vertices, faces, and half-edges.

Sketch-based generation and editing of quad meshes

The disclosure provides an approach for transferring image edits from a source image to target images. In one embodiment, a warp application receives a user-selected region of interest in a source image and determines for the region of interest content-aware bounded weight functions and seed locations for the same. For each of the target images, the warping application initializes a linear blend skinning subspace warp to a projection onto a feature space of a piecewise affine map from scale invariant feature transform features of the source image to the target image. After initializing the warps, the warping application iteratively optimizes the warps by applying the inverse compositional Lucas-Kanade procedure and using the content-aware weight functions in said procedure. Edits made to the source image may automatically be transferred to target images by warping those edits via the optimized warp function for the respective target images.

Transfusive image manipulation

Growing community of inventors

Zurich, Switzerland

Olga Sorkine-Hornung