Adobe, Inc.

Ucl Business Plc

Vladimir Kim

Noam Aigerman

Siddhartha Chaudhuri

Matthew David Fisher

Mehmet Yumer

Oliver Wang

Elya Shechtman

Aaron Phillip Hertzmann

Duygu Ceylan Aksit

Bryan Russell

Emiliano Gambaretto

Jonathan Eisenmann

Alec Jacobson

Hsueh-ti Liu

Radomir Mech

Kalyan Krishna Sunkavalli

Xin Sun

Zhili Chen

Kevin Wampler

Niloy Jyoti Mitra

Jun Saito

Giorgio Gori

Omid Poursaeed

Chen-Hsuan Lin

Pierre-alain Langlois

Jeong Joon Park

Zhaowen Wang

Baldo Antonio Faieta

Byungmoon Kim

Li-Yi Wei

Yannick Hold-Geoffroy

Wilmot Li

Kazi Rubaiat Habib

Duygu Ceylan

Daniel Sýkora

Long Mai

Qingnan Zhou

Aron Monszpart

Minhyuk Sung

Thibault Groueix

Samaneh Azadi

Mengqi Peng

Marek Dvoroznák

Li Yi

Sanjeev Muralikrishnan

Michael Alcorn

Luca Morreale

Kunal Gupta

Bo Sun

Jean-François Lalonde

Mohammad Reza Karimi Dastjerdi

Zimo Li

Methods, systems, and non-transitory computer readable storage media are disclosed that utilizes machine learning models for patch retrieval and deformation in completing three-dimensional digital shapes. In particular, in one or more implementations the disclosed systems utilize a machine learning model to predict a coarse completion shape from an incomplete 3D digital shape. The disclosed systems sample coarse 3D patches from the coarse 3D digital shape and learn a shape distance function to retrieve detailed 3D shape patches in the input shape. Moreover, the disclosed systems learn a deformation for each retrieved patch and blending weights to integrate the retrieved patches into a continuous surface.

Utilizing machine learning models for patch retrieval and deformation in completing three-dimensional digital shapes

Methods, systems, and non-transitory computer readable storage media are disclosed for generating digital chain pull paintings in digital images. The disclosed system generate, utilizing a neural network, a plurality of matrices over an ambient space for a plurality of polygons of a three-dimensional mesh based on a plurality of features of the plurality of polygons associated with the three-dimensional mesh. The disclosed system determines a gradient field based on the plurality of matrices of the plurality of polygons. The disclosed system generates a mapping for the three-dimensional mesh based on the gradient field and a differential operator corresponding to the three-dimensional mesh.

Generating mappings of three-dimensional meshes utilizing neural network gradient fields

Embodiments are disclosed for generating 360-degree panoramas from input narrow field of view images. A method of generating 360-degree panoramas may include obtaining an input image and guide, generating a panoramic projection of the input image, and generating, by a panorama generator, a 360-degree panorama based on the panoramic projection and the guide, wherein the panorama generator is a guided co-modulation generator network trained to generate a 360-degree panorama from the input image based on the guide.

Extrapolating panoramas from images using a generative model

Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing one or more neural networks to recursively subdivide a three-dimensional mesh according to local geometries of vertices in the three-dimensional mesh. For example, the disclosed system can determine a local geometry (e.g., a one-ring neighborhood of half-flaps) for each vertex in a three-dimensional mesh. For each subdivision iteration, the disclosed system can then utilize a neural network to determine displacement coordinates for existing vertices in the three-dimensional mesh and coordinates for new vertices added to edges between the existing vertices in the three-dimensional mesh in accordance with the local geometries of the existing vertices. Furthermore, the disclosed system can generate a subdivided three-dimensional mesh based on the determined displacement coordinates for the existing vertices and the determined coordinates for the new vertices.

Subdividing a three-dimensional mesh utilizing a neural network

Certain aspects and features of this disclosure relate to neural network based 3D object surface mapping. In one example, a first representation of a first surface of a first 3D object and a second representation of a second surface of a second 3D object are produced. A surface mapping function is generated for mapping the first surface to the second surface. The surface mapping function is defined the representations and by a neural network model configured to map a first 2D representation of the first surface to a second 2D representation of the second surface. One or more features of the a first 3D mesh on the first surface can be applied to a second 3D mesh on the second surface using the surface mapping function to produce a modified second surface, which can be rendered through a user interface.

Neural network based 3D object surface mapping

The present disclosure includes methods and systems for identifying and manipulating a segment of a three-dimensional digital model based on soft classification of the three-dimensional digital model. In particular, one or more embodiments of the disclosed systems and methods identify a soft classification of a digital model and utilize the soft classification to tune segmentation algorithms. For example, the disclosed systems and methods can utilize a soft classification to select a segmentation algorithm from a plurality of segmentation algorithms, to combine segmentation parameters from a plurality of segmentation algorithms, and/or to identify input parameters for a segmentation algorithm. The disclosed systems and methods can utilize the tuned segmentation algorithms to accurately and efficiently identify a segment of a three-dimensional digital model.

Utilizing soft classifications to select input parameters for segmentation algorithms and identify segments of three-dimensional digital models

Embodiments of the present invention are directed towards intuitive editing of three-dimensional models. In embodiments, salient geometric features associated with a three-dimensional model defining an object are identified. Thereafter, feature attributes associated with the salient geometric features are identified. A feature set including a plurality of salient geometric features related to one another is generated based on the determined feature attributes (e.g., properties, relationships, distances). An editing handle can then be generated and displayed for the feature set enabling each of the salient geometric features within the feature set to be edited in accordance with a manipulation of the editing handle. The editing handle can be displayed in association with one of the salient geometric features of the feature set.

Intuitive editing of three-dimensional models

Systems and methods for performing image search are described. An image search method may include generating a feature vector for each of a plurality of stored images using a machine learning model trained using a rotation loss term, receiving a search query comprising a search image with object having an orientation, generating a query feature vector for the search image using the machine learning model, wherein the query feature vector is based at least in part on the orientation, comparing the query feature vector to the feature vector for each of the plurality of stored images, and selecting at least one stored image of the plurality of stored images based on the comparison, wherein the at least one stored image comprises a similar orientation to the orientation of the object in the search image.

3D-aware image search

Techniques are described herein for generating and using a unified shape representation that encompasses features of different types of shape representations. In some embodiments, the unified shape representation is a unicode comprising a vector of embeddings and values for the embeddings. The embedding values are inferred, using a neural network that has been trained on different types of shape representations, based on a first representation of a three-dimensional (3D) shape. The first representation is received as input to the trained neural network and corresponds to a first type of shape representation. At least one embedding has a value dependent on a feature provided by a second type of shape representation and not provided by the first type of shape representation. The value of the at least one embedding is inferred based upon the first representation and in the absence of the second type of shape representation for the 3D shape.

Unified shape representation

Certain embodiments involve techniques for generating a 3D representation based on a provided 2D image of an object. An image generation system receives the 2D image representation and generates a multi-dimensional vector of the input that represents the image. The image generation system samples a set of points and provides the set of points and the multi-dimensional vector to a neural network that was trained to predict a 3D surface representing the image such that the 3D surface is consistent with a 3D surface of the object calculated using an implicit function for representing the image. The neural network predicts, based on the multi-dimensional vector and the set of points, the 3D surface representing the object.

Learning hybrid (surface-based and volume-based) shape representation

Mixing segmentation algorithms utilizing soft classifications to identify segments of three-dimensional digital models

Using a prediction engine, generating, based on deformations of prior editing operations performed with a graphics editing tool, suggested editing operations that augment current editing operations applied to a graphical object. The prediction engine accesses first samples defining first positions along first paths of previous editing operations applied to a mesh object in a previous frame and second samples defining second positions along second paths of executed editing operations applied in a current frame. The prediction engine identifies, from a comparison of the first samples and the second samples, a matching component set from the previous editing operations that corresponds to the executed editing operations. The prediction engine deforms the first samples toward the second samples and determines suggested editing operations that comprise a non-matching component set as modified based on the deformed first samples. The prediction engine updates an interface to provide the suggested editing operations.

Generating suggested edits for three-dimensional graphics based on deformations of prior edits

Methods, systems, and non-transitory computer readable storage media are disclosed for iteratively decimating a three-dimensional mesh utilizing successive self-parameterization. For example, the disclosed system can self-parameterize local geometries of a three-dimensional mesh using surface mappings within a two-dimensional surface mapping space. The disclosed system can collapse edges in the three-dimensional mesh to create new vertices from the collapsed edges. The disclosed system can parameterize the collapsed edges based on the surface mappings to collapse corresponding edges within the surface mapping space. The disclosed system can thus generate a decimated three-dimensional mesh by collapsing edges in the three-dimensional mesh while providing a bijective map between points in the decimated three-dimensional mesh and corresponding points in the three-dimensional mesh.

Decimating a three-dimensional mesh via successive self-parameterization

Methods, systems, and non-transitory computer readable storage media are disclosed for reconstructing three-dimensional meshes from two-dimensional images of objects with automatic coordinate system alignment. For example, the disclosed system can generate feature vectors for a plurality of images having different views of an object. The disclosed system can process the feature vectors to generate coordinate-aligned feature vectors aligned with a coordinate system associated with an image. The disclosed system can generate a combined feature vector from the feature vectors aligned to the coordinate system. Additionally, the disclosed system can then generate a three-dimensional mesh representing the object from the combined feature vector.

Reconstructing three-dimensional scenes in a target coordinate system from multiple views

Growing community of inventors

Seattle, WA, United States of America

Vladimir Kim