Disney Enterprises, Inc.

Carnegie Mellon University

Eth Zurich (eidgenossische Technische Hochschule Zurich)

Eth Zurich (eidgenoessische Technische Hochschule Zurich)

Iain Matthews

Yaser Sheikh

Patrick Joseph Lucey

G Peter K Carr

Jose Rafael Tena

Thabo D Beeler

Eric Foote

George Peter Carr

Alina Bialkowski

Robert Walker Sumner

Derek Edward Bradley

Kenneth John Mitchell

Paul A Beardsley

Katsu Yamane

Martin M Rufli

Andreas Breitenmoser

Jean-Francois Lalonde

Roland Siegwart

Moshe Benjamin Mahler

Hengchin Yeh

Cydni Tetro

John-Thomas C Ngo

Rajitha Navarathna

Daniel Candela

Sarah Taylor

Tomas Simon

G Peter Carr

Scott Frazier Watson

Matthias Grundmann

Tunc Ozan Aydin

Jon Hayes Snoddy

Irfan Essa

Jessica Kate Hodgins

Long Sha

Alfredo Medina Ayala

Fernando De La Torre

Ariel Shamir

Charalampos Koniaris

Yisong Yue

Darren Cosker

Javier Alonso Mora

Ijaz Akhter

Steven G McDonagh

Martin Klaudiny

Paulo Urnau Gotardo

Anthony D Martin

Kihwan Kim

Peter Carr

Xiasong Yang

Miguel Granados

Jean Francois Lalonde

Michael Peter K Mistry

Christian Theobalt

Natasha Kholgade

Sohaib Khan

Barry John Theobald

Barry-John Theobald

Margaret Meyerhofer

A system and method generates a broadcast image. The method includes receiving a first image captured by a first camera having a first configuration incorporating a first center of projection. The method includes determining a first mapping from a first image plane of the first camera onto a sphere. The method includes determining a second mapping from the sphere to a second image plane of a second virtual camera having a second configuration incorporating the first center of projection of the first configuration. The method includes generating a second image based upon the first image and a concatenation of the first and second mappings.

Method and device for hybrid robotic/virtual pan-tilt-zoom cameras for autonomous event recording

Provided are methods, systems, and computer-readable medium for synthetically generating training data to be used to train a learning algorithm that is capable of generating computer-generated images of a subject from real images that include the subject. The training data can be generated using a facial rig by changing expressions, camera viewpoints, and illumination in the training data. The training data can then be used for tracking faces in a real-time video stream. In such examples, the training data can be tuned to expected environmental conditions and camera properties of the real-time video stream. Provided herein are also strategies to improve training set construction by analyzing which attributes of a computer-generated image (e.g., expression, viewpoint, and illumination) require denser sampling.

Markerless face tracking with synthetic priors

Approaches are described for formation retrieval. Embodiments receive positional data, across an interval window, including a respective agent trajectory for each agent and an object trajectory for one or more objects. The interval window is partitioned into frames and, at each frame, embodiments calculate a cost of assigning a role to each agent based on one or more exemplar formations. A formation is determined by assigning a role to each agent based on the calculated cost. Each frame of the formation is compared to a corresponding frame of a stored formation, by calculating a distance between a position of each assigned role in the frame and a position of a corresponding role in the stored formation and by comparing the object trajectory for the one or more objects with a corresponding object trajectory in the stored formation. Based on the comparisons, a list of stored formations is generated.

Sports formation retrieval

Approaches are described for discovering a formation associated with an agent group engaging in an activity over a window of time. A formation analysis system computes first and second results for an objective function based on first and second sets of role assignments for each agent in the agent group at first and second moments in time, respectively. The formation analysis system iterates by: replacing the first set of role assignments with the second set of role assignments, and determining whether completion criteria have been met based at least in part on comparing the first result with the second result. If the completion criteria have not been met, then the formation analysis system replaces the second set of role assignments with a third set of role assignments that associate each agent in the first agent group with a different role assignment in the third set of role assignments at a third moment in time. If the completion criteria have been met, then the formation analysis system determines the first formation based on the second result.

Analysis of team behaviors using role and formation information

There are provided systems and methods for generating a visually consistent alternative audio for redubbing visual speech using a processor configured to sample a dynamic viseme sequence corresponding to a given utterance by a speaker in a video, identify a plurality of phonemes corresponding to the dynamic viseme sequence, construct a graph of the plurality of phonemes that synchronize with a sequence of lip movements of a mouth of the speaker in the dynamic viseme sequence, use the graph to generate an alternative phrase that substantially matches the sequence of lip movements of the mouth of the speaker in the video.

Generating a visually consistent alternative audio for redubbing visual speech

Speech animation may be performed using visemes with phonetic boundary context. A viseme unit may comprise an animation that simulates lip movement of an animated entity. Individual ones of the viseme units may correspond to one or more complete phonemes and phoneme context of the one or more complete phonemes. Phoneme context may include a phoneme that is adjacent to the one or more complete phonemes that correspond to a given viseme unit. Potential sets of viseme units that correspond with individual phoneme string portions may be determined. One of the potential sets of viseme units may be selected for individual ones of the phoneme string portions based on a fit metric that conveys a match between individual ones of the potential sets and the corresponding phoneme string portion.

Systems and methods for speech animation using visemes with phonetic boundary context

A system and method improves a camera calibration. The method includes receiving a camera image, a planar template pattern, a 3D geometry of a surface on which the planar template pattern is embedded, and a set of parameter values. The method includes rendering the planar template pattern into a camera perspective based on the parameter values to generate a warped template image. The method includes generating an error image including at least one non-zero difference between the camera image and the warped template image. The method includes adjusting the parameter values to reduce an error between the camera image and the warped template image.

Method and system for determining camera parameters from a long range gradient based on alignment differences in non-point image landmarks

Embodiments disclosed herein relate to a method and apparatus for generating a three-dimensional surface. In one embodiment, there is a method for generating a three-dimensional surface. The method includes capturing a plurality of images of a target object with at least two cameras, the target object illuminated by at least two sets of red-green-blue (RGB) lights positioned in an array about the target object and generating a three-dimensional surface of the target object by iteratively reconstructing a surface estimate of the target object and aligning images of the target object using motion estimation until the images converge, wherein the images are processed in n-frame intervals.

Photometric three-dimensional facial capture and relighting

Methods and systems for estimating HDR sky light probes for outdoor images are disclosed. A precaptured sky light probe database is leveraged. The database includes a plurality of HDR sky light probes captured under a plurality of different illumination conditions. A HDR sky light probe is estimated from an outdoor image by fitting a three dimensional model to an object of interest in the image and solving an inverse optimization lighting problem for the 3D model where the space of possible HDR sky light probes is constrained by the HDR sky light probes of the database.

Systems and methods for estimating sky light probes for outdoor images

There is provided a method for use by a system including a hardware processor and a memory storing a three-dimensional (3D) character data having a topology mesh describing a character. The method includes stabilizing the 3D character data by removing a translational motion and a rotational motion from the 3D character data, determining a range of motion of a plurality of points of the topology mesh describing the character, identifying a plurality of poses for generating an animation, where each of the plurality of poses includes a unique configuration of the plurality of points of the topology mesh, and creating an animatronic including a plurality of actuators configured to instantiate the plurality of poses for generating the animation.

Data driven design and animation of animatronics

Techniques are described to characterize motion patterns of a group of agents engaging in an activity. An analysis system receives input data associated with spatial and temporal information of at least one element of interest associated with the activity, where the object of interest may be a ball, person, animal or any other object in motion. The analysis system partitions the input data into a plurality of spatiotemporal segments and generates one or more representations of one or more sets of segments of the plurality of spatiotemporal segments based on one or more criteria. The analysis system computes a metric, such as an entropy value, for each of the one or more representations. Partial tracing data, such as ball movements in a sporting event, may be created using an inexpensive input device, such as a tablet computer, making the disclosed techniques available for a wide range of events and activities.

Characterizing motion patterns of one or more agents from spatiotemporal data

Techniques are disclosed for creating digital assets that can be used to personalize themed products. For example, a workflow and pipeline used to generate a 3D model from digital images of a person's face and to manufacture a personalized, physical figurine customized with the 3D model are disclosed. The 3D model of the person's face may be simplified to match a topology of a desired figurine. While the topology is deformed to match that of the figurine, the 3D model retains the geometry of the child's face. Simplifying the topology of the 3D model in this manner allows the mesh to be integrated with or attached to a mesh representing desired figurine.

Techniques for processing reconstructed three-dimensional image data

To generate a media presentation of a live event, a user interface is coupled to at least three cameras that share substantially the same vantage point. One of the cameras (e.g., a context camera) provides a context view of the event that is displayed on a screen of the user interface. The views of the other two cameras are superimposed onto the context view to define sub-portions that are visually demarcated within the context view. Based on user interaction, the user interface can switch between the cameras views and control the cameras to capture different portions of the context view. Based the image data captured by the views of the cameras within the context view, the user interface generates a media presentation that may be broadcast to multiple viewers.

Camera selection interface for producing a media presentation

Methods and systems for predicting light probes for outdoor images are disclosed. A light probe database is created to learn a mapping from the outdoor image's features to predicted outdoor light probe illumination parameters. The database includes a plurality of images, image features for each of the plurality of images, and a captured light probe for each of the plurality of images. A light probe illumination model based on a sun model and sky model is fitted to the captured light probes. The light probe for the outdoor image may be predicted based on the database dataset and fitted light probe models.

Predicting a light probe for an outdoor image

Described herein are methods, systems, apparatuses and products for utilizing motion fields to predict evolution in dynamic scenes. One aspect provides for accessing active object position data including positioning information of a plurality of individual active objects; extracting a plurality of individual active object motions from the active object position data; constructing a motion field using the plurality of individual active object motions; and using the motion field to predict one or more points of convergence at one or more spatial locations that active objects are proceeding towards at a future point in time. Other embodiments are disclosed.

System and method for utilizing motion fields to predict evolution in dynamic scenes

Growing community of inventors

Pittsburgh, PA, United States of America

Iain Matthews