Disney Enterprises, Inc.

Eth Zurich (eidgenossische Technische Hochschule Zurich)

Eidgenossische Technische Hochschule Zurich

Eth Zurich (eidgenoessische Technische Hochschule Zurich)

Maurizio Nitti

Robert Walker Sumner

Mattia Ryffel

Alessia Marra

Stephane Magnenat

Dat Tien Ngo

Gioacchino Noris

Markus Gross

Stelian Coros

Bernhard Thomaszewski

Kenny Mitchell

Martin Guay

Jakob Joachim Buhmann

Steven Poulakos

Fabio Zünd

Dominik Tobias Borer

Fabian Andreas Hahn

Gerhard Roethlin

Alba Maria Rios Rodriguez

Frederik Rudolf Mutzel

In various embodiments, a training application generates training items for three-dimensional (3D) pose estimation. The training application generates multiple posed 3D models based on multiple 3D poses and a 3D model of a person wearing a costume that is associated with multiple visual attributes. For each posed 3D model, the training application performs rendering operation(s) to generate synthetic image(s). For each synthetic image, the training application generates a training item based on the synthetic image and the 3D pose associated with the posed 3D model from which the synthetic image was rendered. The synthetic images are included in a synthetic training dataset that is tailored for training a machine-learning model to compute estimated 3D poses of persons from two-dimensional (2D) input images. Advantageously, the synthetic training dataset can be used to train the machine-learning model to accurately infer the orientations of persons across a wide range of environments.

Techniques for inferring three-dimensional poses from two-dimensional images

Techniques for animation are provided. A first trajectory for a first element in a first animation is determined. A first approximation is generated based on the first trajectory, and the first approximation is modified based on an updated state of the first element. The first trajectory is then refined based on the modified first approximation.

Parameterized animation modifications

In particular embodiments, a 2D representation of an object may be provided. A first method may comprise: receiving sketch input identifying a target position for a specified portion of the object; computing a deformation for the object within the context of a character rig specification for the object; and displaying an updated version of the object. A second method may comprise detecting sketch input; classifying the sketch input, based on the 2D representation, as an instantiation of the object; instantiating the object using a 3D model of the object; and displaying a 3D visual representation of the object.

Sketch-based abstraction for character posing and synthesis

There are provided systems and methods for performing deformable-surface tracking based augmented reality image generation. In one implementation, such a system includes a hardware processor and a system memory storing an augmented reality three-dimensional image generator. The hardware processor is configured to execute the augmented reality three-dimensional image generator to receive image data corresponding to a two-dimensional surface, and to identify an image template corresponding to the two-dimensional surface based on the image data. In addition, the hardware processor is configured to execute the augmented reality three-dimensional image generator to determine a surface deformation of the two-dimensional surface. The hardware processor is further configured to execute the augmented reality three-dimensional image generator to generate an augmented reality three-dimensional image including at least one feature of the two-dimensional surface, based on the image template and the surface deformation of the two-dimensional surface.

Deformable-surface tracking based augmented reality image generation

Systems and methods for re-arranging a musical composition using augmented reality are disclosed. A user can be provided with one or more physical image markers representative of at least one of a musical entity and a musical style in which a musical track has been pre-recorded. Upon capturing a visual scene in which one or more of these physical image markers are included, an augmented reality version of the visual scene is displayed along with audio corresponding to the musical style(s) and/or entities represented by the one or more physical image markers.

Augmented reality music composition

There is provided a system and method for augmented reality image transformation. In one implementation, such a system includes a system processor, a system memory, and an image transformation engine stored in the system memory. The system processor is configured to execute the image transformation engine to receive image data corresponding to an image, to detect a feature on a surface of the image, and to synthesize a texture corresponding to the feature. The system processor is further configured to execute the image transformation engine to produce an augmented reality representation of the image having the texture for display to a user, the texture covering a surface of the augmented reality representation of the image corresponding to a portion of the feature that is not visible on the surface of the image.

Augmented reality image transformation

Growing community of inventors

Zurich, Switzerland

Maurizio Nitti