Disney Enterprises, Inc.

Eth Zurich (eidgenoessische Technische Hochschule Zurich)

Eth Zurich

Eth Zurich (eidgenossische Technische Hochschule Zurich)

Pixar

Eidgenossische Technische Hochschule Zurich

Robert Walker Sumner

Stelian Coros

Mattia Ryffel

Markus H Gross

Martin Guay

Maurizio Nitti

Gioacchino Noris

Thabo D Beeler

Alexander Sorkine Hornung

Markus Gross

Alessia Marra

Bernhard Thomaszewski

Kenneth John Mitchell

Bernd Bickel

Paul A Beardsley

Kenny Mitchell

Mubbasir Kapadia

Thomas Oskam

Maryann Simmons

Steven Poulakos

Dominik Tobias Borer

Fabian Hahn

Sebastian Martin

Johannes Schmid

Christoph Maurhofer

Stephane Magnenat

Iain Matthews

Wojciech Jarosz

Ilya Baran

Brian Whited

Moshe Benjamin Mahler

Jakob Joachim Buhmann

Fabio Zünd

Jose Rafael Tena

Ye Yuan

Gökcen Cimen

Cydni Tetro

Hengchin Yeh

John-Thomas C Ngo

Rafael C Hostettler

Manuel Braunschweiler

Alexander Shoulson

Dat Tien Ngo

Daniel Sykora

Anthony M Accardo

Mark Joseph Meyer

Alanson P Sample

Thomas R Gross

Derek Nowrouzezahrai

Ariel Shamir

Manuel Lang

Ahmet Cengiz Öztireli

Alexandre Chapiro

Huw David Bowles

Steve Marschner

Forrester Cole

Gerhard Roethlin

Seth Frey

Fabian Andreas Hahn

Thomas Williams

Anthony Derose

Marie-Paule Cani

Antoine Milliez

Martin Sebastian Senn

Loic Ciccone

Raphael Anderegg

Stefan C Geiger

Huw Bowles

Rafael Huber

Simone Guggiari

Gökçen Çimen

Moritz Geilinger

Mubbasir Turab Kapadia

Xu Xianghao

Frederik Rudolf Mutzel

Virag Varga

Gergely Vakulya

Benjamin Buergisser

Alba Maria Rios Rodriguez

An image processing system includes a computing platform having a hardware processor and a system memory storing an image augmentation software code, a three-dimensional (3D) shapes library, and/or a 3D poses library. The image processing system also includes a two-dimensional (2D) pose estimation module communicatively coupled to the image augmentation software code. The hardware processor executes the image augmentation software code to provide an image to the 2D pose estimation module and to receive a 2D pose data generated by the 2D pose estimation module based on the image. The image augmentation software code identifies a 3D shape and/or a 3D pose corresponding to the image using an optimization algorithm applied to the 2D pose data and one or both of the 3D poses library and the 3D shapes library, and may output the 3D shape and/or 3D pose to render an augmented image on a display.

Automated image augmentation using a virtual character

Techniques are disclosed for learning a machine learning model that maps control data, such as renderings of skeletons, and associated three-dimensional (3D) information to two-dimensional (2D) renderings of a character. The machine learning model may be an adaptation of the U-Net architecture that accounts for 3D information and is trained using a perceptual loss between images generated by the machine learning model and ground truth images. Once trained, the machine learning model may be used to animate a character, such as in the context of previsualization or a video game, based on control of associated control points.

Techniques for feature-based neural rendering

A storytelling system includes location-based content (LBC) servers and LBC effects databases for use in a real-world story venue, and a computing platform communicatively coupled to those LBC servers and databases. A processor of the computing platform executes software code to receive story selection data from a user device, obtain a story template including story arcs each associated with one or more LBC effect(s) that corresponds to the story selection data, and determine, using a location of the user device, one of the story arcs as an active story arc. The software code also identifies an LBC interaction zone for the LBC effect(s), designates one LBC server and one LBC effects database for supporting the active story arc, and distributes one or more of the LBC effect(s) to the designated database. The designated server enables instantiation of the one or more of the LBC effect(s) at the LBC interaction zone.

Location-based interactive storytelling

A virtual puppeteering system includes a portable device including a camera, a display, a hardware processor, and a system memory storing an object animation software code. The hardware processor is configured to execute the object animation software code to, using the camera, generate an image in response to receiving an activation input, using the display, display the image, and receive a selection input selecting an object shown in the image. The hardware processor is further configured to execute the object animation software code to determine a distance separating the selected object from the portable device, receive an animation input, identify, based on the selected object and the received animation input, a movement for animating the selected object, generate an animation of the selected object using the determined distance and the identified movement, and render the animation of the selected object.

Virtual puppeteering using a portable device

Techniques for animation are provided. A first trajectory for a first element in a first animation is determined. A first approximation is generated based on the first trajectory, and the first approximation is modified based on an updated state of the first element. The first trajectory is then refined based on the modified first approximation.

Parameterized animation modifications

This disclosure presents systems and methods to provide an interactive environment in response to touch-based inputs. A first body channel communication device coupled to a user may transmit and/or receive signals configured to be propagated along skin of the user such that the skin of the user comprises a signal transmission path. A second body channel communication device coupled to an interaction entity may be configured to transmit and/or receive signals configured to be propagated along the skin of the user along the signal transmission path. A presentation device may present images of virtual content to the user. Information may be communicated between the first body channel communication device, the second body channel communication device, and the presentation device so that virtual content specific to the interaction entity may be presented to augment an appearance of the interaction entity.

Systems and methods to provide an interactive environment in response to touch-based inputs

In particular embodiments, a 2D representation of an object may be provided. A first method may comprise: receiving sketch input identifying a target position for a specified portion of the object; computing a deformation for the object within the context of a character rig specification for the object; and displaying an updated version of the object. A second method may comprise detecting sketch input; classifying the sketch input, based on the 2D representation, as an instantiation of the object; instantiating the object using a 3D model of the object; and displaying a 3D visual representation of the object.

Sketch-based abstraction for character posing and synthesis

Techniques are disclosed for estimating poses from images. In one embodiment, a machine learning model, referred to herein as the 'detector,' is trained to estimate animal poses from images in a bottom-up fashion. In particular, the detector may be trained using rendered images depicting animal body parts scattered over realistic backgrounds, as opposed to renderings of full animal bodies. In order to make appearances of the rendered body parts more realistic so that the detector can be trained to estimate poses from images of real animals, the body parts may be rendered using textures that are determined from a translation of rendered images of the animal into corresponding images with more realistic textures via adversarial learning. Three-dimensional poses may also be inferred from estimated joint locations using, e.g., inverse kinematics.

Joint estimation from images

An automated costume augmentation system includes a computing platform having a hardware processor and a system memory storing a software code. The hardware processor executes the software code to provide an image including a posed figure to an artificial neural network (ANN), receive from the ANN a 2D skeleton data including joint positions corresponding to the posed figure, and determine a 3D pose corresponding to the posed figure using an optimization algorithm applied to the skeleton data. The software code further identifies one or more proportion(s) of the posed figure based on the skeleton data, determines bone directions corresponding to the posed figure using another optimization algorithm applied to the 3D pose, parameterizes a costume for the posed figure based on the 3D pose, the proportion(s), and the bone directions, and outputs an enhanced image including the posed figure augmented with the fitted costume for rendering on a display.

Automated costume augmentation using shape estimation

A user interface may be presented to a user that may provide an intuitive interface configured for generating animations. The user interface may be users of varying levels of expertise, while retaining the freedom of authoring complex narratives in an animation. A given narrative may be constrained by user input related to one or more of events to occur in an animation, individual animation components that may be included in the events, and/or other input. The system may be configured to 'fill in' missing gaps in a narrative to generate a consistent animation while still meeting one or more narrative constraints specified by user input. By way of non-limiting example, gaps may be 'filled in' by effectuating non-user selections of one or more of events, animation components, and/or other information used to generate an animation.

Systems and methods for generating event-centric animations using a graphical user interface

There are provided systems and methods for performing deformable-surface tracking based augmented reality image generation. In one implementation, such a system includes a hardware processor and a system memory storing an augmented reality three-dimensional image generator. The hardware processor is configured to execute the augmented reality three-dimensional image generator to receive image data corresponding to a two-dimensional surface, and to identify an image template corresponding to the two-dimensional surface based on the image data. In addition, the hardware processor is configured to execute the augmented reality three-dimensional image generator to determine a surface deformation of the two-dimensional surface. The hardware processor is further configured to execute the augmented reality three-dimensional image generator to generate an augmented reality three-dimensional image including at least one feature of the two-dimensional surface, based on the image template and the surface deformation of the two-dimensional surface.

Deformable-surface tracking based augmented reality image generation

A method of identifying locations in a virtual environment where a motion sequence can be performed by an animated character may include accessing the motion sequence for the animated character, identifying a plurality of contact locations in the motion sequence where the animated character contacts surfaces in virtual environments, accessing the virtual environment comprising a plurality of surfaces, and identifying the locations in the virtual environment where the motion sequence can be performed by the animated character by identifying surfaces in the plurality of surfaces that match the plurality of contact locations.

Precomputed environment semantics for contact-rich character animation

Techniques for generating locomotion data for animating a virtual quadruped model, starting at an origin point and travelling to a destination point along a defined path. A virtual skeletal structure for the virtual quadruped model is analyzed to identify a torso region, limbs each ending in a respective end effector, and limb attributes. A predefined locomotion template for virtual quadruped characters is retrieved and mapped to the virtual quadruped model by aligning the torso region and the plurality of limbs of the virtual skeletal structure for the virtual quadruped with a second torso region and a second plurality of limbs of the predefined locomotion template. Locomotion data is generated for the virtual quadruped model based on the defined path and by upscaling the mapped predefined locomotion template, based at least on the set of limb attributes determined by analyzing the virtual skeletal structure for the virtual quadruped model.

Automatically generating quadruped locomotion controllers

Growing community of inventors

Zurich, Switzerland

Robert Walker Sumner