Adobe, Inc.

Ucl Business Plc

Noam Aigerman

Vladimir Kim

Siddhartha Chaudhuri

Alec Jacobson

Hsueh-ti Liu

Radomir Mech

Duygu Ceylan Aksit

Emiliano Gambaretto

Kevin Wampler

Jonathan Eisenmann

Giorgio Gori

Silvia Gonzalez Sellan

Matthew David Fisher

Niloy Jyoti Mitra

Jun Saito

Omid Poursaeed

Thibault Groueix

Luca Morreale

Kunal Gupta

Bo Sun

Methods, systems, and non-transitory computer readable storage media are disclosed that utilizes machine learning models for patch retrieval and deformation in completing three-dimensional digital shapes. In particular, in one or more implementations the disclosed systems utilize a machine learning model to predict a coarse completion shape from an incomplete 3D digital shape. The disclosed systems sample coarse 3D patches from the coarse 3D digital shape and learn a shape distance function to retrieve detailed 3D shape patches in the input shape. Moreover, the disclosed systems learn a deformation for each retrieved patch and blending weights to integrate the retrieved patches into a continuous surface.

Utilizing machine learning models for patch retrieval and deformation in completing three-dimensional digital shapes

Methods, systems, and non-transitory computer readable storage media are disclosed for generating digital chain pull paintings in digital images. The disclosed system generate, utilizing a neural network, a plurality of matrices over an ambient space for a plurality of polygons of a three-dimensional mesh based on a plurality of features of the plurality of polygons associated with the three-dimensional mesh. The disclosed system determines a gradient field based on the plurality of matrices of the plurality of polygons. The disclosed system generates a mapping for the three-dimensional mesh based on the gradient field and a differential operator corresponding to the three-dimensional mesh.

Generating mappings of three-dimensional meshes utilizing neural network gradient fields

Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing one or more neural networks to recursively subdivide a three-dimensional mesh according to local geometries of vertices in the three-dimensional mesh. For example, the disclosed system can determine a local geometry (e.g., a one-ring neighborhood of half-flaps) for each vertex in a three-dimensional mesh. For each subdivision iteration, the disclosed system can then utilize a neural network to determine displacement coordinates for existing vertices in the three-dimensional mesh and coordinates for new vertices added to edges between the existing vertices in the three-dimensional mesh in accordance with the local geometries of the existing vertices. Furthermore, the disclosed system can generate a subdivided three-dimensional mesh based on the determined displacement coordinates for the existing vertices and the determined coordinates for the new vertices.

Subdividing a three-dimensional mesh utilizing a neural network

Certain aspects and features of this disclosure relate to neural network based 3D object surface mapping. In one example, a first representation of a first surface of a first 3D object and a second representation of a second surface of a second 3D object are produced. A surface mapping function is generated for mapping the first surface to the second surface. The surface mapping function is defined the representations and by a neural network model configured to map a first 2D representation of the first surface to a second 2D representation of the second surface. One or more features of the a first 3D mesh on the first surface can be applied to a second 3D mesh on the second surface using the surface mapping function to produce a modified second surface, which can be rendered through a user interface.

Neural network based 3D object surface mapping

Techniques for determining a swept volume of an object moving along a trajectory in a 3D space are disclosed. In some examples, a computer graphics application accesses a representation of the object, such as the signed distance field (SDF), and the trajectory information describing the movement path in the 3D space over a time period. The 3D space is represented using a grid of voxels each having multiple vertices. The computer graphics application determines the swept volume of the object in the 3D space by evaluating a subset of the grid of voxels (e.g., the voxels surrounding the surface of the swept volume). The number of voxels in the subset of voxels is less than the number of voxels in the grid of voxels. The computer graphics application further generates a representation of the swept volume surface for output.

Swept volume determination techniques

Three-dimensional (3D) mesh segmentation techniques are described. In one example, a geometry segmentation system determines a vertex direction for each vertex in a 3D mesh. A segment generation module is then employed to generate segments (e.g., as developable geometries) from the 3D mesh. To do so, a vertex selection module selects an initial vertex having an associated vertex direction. A face identification module then identifies a face in the 3D mesh using that initial vertex and at least one other vertex. A segment determination module compares the vertex direction associated with the initial vertex with a normal determined for the face. If the vertex direction is orthogonal to the normal (e.g., within a threshold amount), the face is added to the segment, and sets another one of the vertices of the face as the initial vertex and the process repeats.

Three-dimensional mesh segmentation

Embodiments of the present invention are directed towards intuitive editing of three-dimensional models. In embodiments, salient geometric features associated with a three-dimensional model defining an object are identified. Thereafter, feature attributes associated with the salient geometric features are identified. A feature set including a plurality of salient geometric features related to one another is generated based on the determined feature attributes (e.g., properties, relationships, distances). An editing handle can then be generated and displayed for the feature set enabling each of the salient geometric features within the feature set to be edited in accordance with a manipulation of the editing handle. The editing handle can be displayed in association with one of the salient geometric features of the feature set.

Intuitive editing of three-dimensional models

Certain embodiments involve techniques for generating a 3D representation based on a provided 2D image of an object. An image generation system receives the 2D image representation and generates a multi-dimensional vector of the input that represents the image. The image generation system samples a set of points and provides the set of points and the multi-dimensional vector to a neural network that was trained to predict a 3D surface representing the image such that the 3D surface is consistent with a 3D surface of the object calculated using an implicit function for representing the image. The neural network predicts, based on the multi-dimensional vector and the set of points, the 3D surface representing the object.

Learning hybrid (surface-based and volume-based) shape representation

Methods, systems, and non-transitory computer readable storage media are disclosed for iteratively decimating a three-dimensional mesh utilizing successive self-parameterization. For example, the disclosed system can self-parameterize local geometries of a three-dimensional mesh using surface mappings within a two-dimensional surface mapping space. The disclosed system can collapse edges in the three-dimensional mesh to create new vertices from the collapsed edges. The disclosed system can parameterize the collapsed edges based on the surface mappings to collapse corresponding edges within the surface mapping space. The disclosed system can thus generate a decimated three-dimensional mesh by collapsing edges in the three-dimensional mesh while providing a bijective map between points in the decimated three-dimensional mesh and corresponding points in the three-dimensional mesh.

Decimating a three-dimensional mesh via successive self-parameterization

An image processing system receives an input depth image with a surface that is not developable and generates an output depth image with a piecewise developable surface that approximates the input depth image. Height values for the output depth image are determined using an optimization problem that balances data fidelity and developability. Data fidelity is based on minimizing differences in height values of pixels in the output depth image and height values of pixels in the input depth image. Developability is based on rank minimization of Hessians computed for pixels in the output depth image. In some configurations, the optimization problem is formulated as a semi-definite programming problem and solved using a tailor-made alternating direction method of multipliers algorithm.

Growing community of inventors

San Francisco, CA, United States of America

Noam Aigerman