Sony Corporation

Sony Pictures Entertainment, Inc.

Universal City Studios, Inc.

Dreamworks Animation LLC

Demian Gordon

Dennis J Hauck, Jr

Parag Havaldar

Dennis J Hauck

Remington Scott

Jerome Chen

Albert Robert Hastings

Jody Echegaray

Josh T Wills

Bruce E Dobrin

Emmanuel C Francisco

Josh Ochoa

Jianxia Xue

Sarah Scialli

Elvin Korkuti

Albert Robert c/o Sony Electronics Inc Hastings

Jody c/o Sony Electronics Inc Echegaray

Systems and methods for controlling performance of a digital character depicted at a display device are disclosed. According to at least one embodiment, a method for controlling performance of a digital character depicted at a display device includes: determining a presence of a person located in a physical environment; and in response to determining the presence of the person, facilitating control of the performance of the digital character depicted at the display device by a human operator, by an artificial intelligence (AI) game-engine, or by a combination thereof.

Digital character interacting with customer in physical realm

A method for evaluating a facial performance using facial capture of two users includes obtaining a reference set of facial performance data representing a first user's facial capture; obtaining a facial capture of a second user; extracting a second set of facial performance data based on the second user's facial capture; calculating at least one matching metric based on a comparison of the reference set of facial performance data to the second set of facial performance data; and displaying an indication of the at least one matching metric on a display.

Facial capture analysis and training system

Motion capturing markers coupled to at least one motion capture object, comprising: generating a plurality of Configurations, each Configuration including partitioning parameters; partitioning a motion capture beat into beat partitions using the plurality of Configurations; reconstructing the beat partitions, wherein each beat partition is reconstructed using each Configuration and corresponding motion capture camera calibration information to generate point data representing a marker; and merging the point data to form at least one master point file.

Reconstruction render farm used in motion capture

Animating digital characters based on motion captured performances, including: receiving sensory data collected using a variety of collection techniques including optical video, electro-oculography, and at least one of optical, infrared, and inertial motion capture; and managing and combining the collected sensory data to aid cleaning, tracking, labeling, and re-targeting processes. Keywords include Optical Video Data and Inertial Motion Capture.

Combining multi-sensory inputs for digital animation

Capturing a surface in motion picture, including: covering a surface with a pattern formed of a marking material; acquiring a sequence of image frames, each image frame of the sequence including a plurality of images of the pattern covering the surface; deriving a mesh object from the plurality of images for the each image frame; tracking the mesh object in each frame through the sequence of frames; and generating animation data modeling a characteristic of the surface using the tracked mesh object.

Capturing surface in motion picture

Tracking a group of motion capture markers in a sequence of frames, comprising: establishing a group relationship among a plurality of motion capture markers in a current frame; and labeling the plurality of motion capture markers in the current frame using the group relationship.

Group tracking in motion capture

Capturing and processing facial motion data includes: coupling a plurality of sensors to target points on a facial surface of an actor; capturing frame by frame images of the plurality of sensors disposed on the facial surface of the actor using at least one motion capture camera disposed on a head-mounted system; performing, in the head-mounted system, a tracking function on the frame by frame images of the plurality of sensors to accurately map the plurality of sensors for each frame; and generating, in the head-mounted system, a modeled surface representing the facial surface of the actor.

Capturing and processing facial motion data

Capturing motion, including: coupling at least one body marker on at least one body point of at least one actor; coupling at least one facial marker on at least one facial point of the at least one actor; arranging a plurality of motion capture cameras around a periphery of a motion capture volume, the plurality of motion capture cameras is arranged such that substantially all laterally exposed surfaces of the at least one actor while in motion within the motion capture volume are within a field of view of at least one of the plurality of motion capture cameras at substantially all times; attaching at least one wearable motion capture camera to the at least one actor, wherein substantially all of the at least one facial marker and the at least one body marker on the at least one actor are within a field of view of the at least one wearable motion capture camera.

Mobile motion capture cameras

Capturing the motion of a target. One method includes: coupling a plurality of primary markers to the target; coupling at least one secondary marker to the target; capturing a plurality of primary marker data points, wherein each primary marker data point corresponds to one primary marker of the plurality of primary markers; capturing at least one secondary marker signature, each secondary marker signature corresponding to and uniquely identifying each secondary marker of said at least one secondary marker; and identifying the plurality of primary markers using said at least one secondary marker signature.

Motion capture using primary and secondary markers

Capturing motion using motion capture cameras comprises: coupling a plurality of markers to an actor; allowing a material to be positioned between the actor and the motion capture cameras, wherein the material is selected to allow the motion capture cameras to capture motion of the plurality of markers while the actor interacts with the material; and capturing the motion of the markers.

Material for motion capture costumes and props

Labeling motion capture volume data includes: receiving a motion capture beat, the motion capture beat including unlabeled body points and unlabeled face points; labeling the unlabeled body points which have a valid fit to a predetermined body model template; verifying the labeled body points using stretch analysis; labeling unlabeled body points using a kinematic skeleton analysis; isolating the unlabeled face points; stabilizing the unlabeled face points; labeling the unlabeled face points; and merging the labeled face points and labeled body points.

Labeling used in motion capture

A method of cleaning facial marker data, the method comprising: decimating frames of survey facial marker data into representative frames of key facial poses; generating a facial pose matrix using the representative frames of key facial poses; and cleaning incoming frames of facial marker data using the facial pose matrix.

FACS cleaning in motion capture

A system for capturing motion comprises: a motion capture volume configured to include at least one moving object having markers defining a plurality of points on the at least one moving object; at least one mobile motion capture camera, the at least one mobile motion capture camera configured to be moveable within the motion capture volume; and a motion capture processor coupled to the at least one mobile motion capture camera to produce a digital representation of movement of the at least one moving object.

A quantum nanodot camera, including: a quantum nanodot camera sensor including: at least one visible pixel sensor configured to capture scenes including actors and/or objects in a visible band; and at least one IR pixel sensor configured to capture motions of at least one quantum nanodot (QD) marker tuned to emit a narrowband IR signal.

Capturing motion using quantum nanodot sensors

Replacing eye movement data in motion capture, comprising: capturing eye movement data from a first actor subsequent to a motion capture session where motion data for a second actor was captured, wherein the first actor performs to generate only the eye movement data with minimal movement of facial muscles that generate interfering signals; and representing the eye movement data captured from the first actor as eye movement data for the second actor in combination with motion data obtained during the motion capture session.

Growing community of inventors

Culver City, CA, United States of America

Demian Gordon