Magic Leap, Inc.

Sony Computer Entertainment America LLC

Other

Thomas Marshall Miller, Iv

The Hung Quach

John Monos

Jeffrey Lin

Takashi Kuribayashi

Lorena Pazmino

Karen Stolzenberg

Gregory Minh Tran

Cole Parker Heiner

Frank Alexander Hamilton, Iv

Victor Ng-Thow-Hing

Sean Michael Comer

Rodrigo Cano

James Jonathan Bancroft

Josh Anon

Nafees Bin Zafar

Stephane Antoine Joseph Imbert

Anthony Marinello

Jennifer M R Devine

Joelle Zimmermann

A user may interact and view virtual elements such as avatars and objects and/or real world elements in three-dimensional space in an augmented reality (AR) session. The system may allow one or more spectators to view from a stationary or dynamic camera a third person view of the users AR session. The third person view may be synchronized with the user view and the virtual elements of the user view may be composited onto the third person view.

Spectator view of virtual and physical objects

Examples of systems and methods for rendering an avatar in a mixed reality environment are disclosed. The systems and methods may be configured to automatically scale an avatar or to render an avatar based on a determined intention of a user, an interesting impulse, environmental stimuli, or user saccade points. The disclosed systems and methods may apply discomfort curves when rendering an avatar. The disclosed systems and methods may provide a more realistic interaction between a human user and an avatar.

Contextual-based rendering of virtual avatars

Systems and methods for displaying a virtual character in a mixed reality environment are disclosed. In some embodiments, a view of the virtual character is based on an animation rig comprising primary joints and helper joints. The animation rig may be in a pose defined by spatial relationships between the primary joints and helper joints. The virtual character may be moving in the mixed reality environment. In some instances, the virtual character may be moving based on a comparison of interestingness values associated with elements in the mixed reality environment. The spatial relationship transformation associated with the movement may be indicated by movement information. In some embodiments, the movement information is received from a neural network.

Photoreal character configurations for spatial computing

Examples of the disclosed systems and methods may provide for improved and more realistic rendering of virtual characters and a more realistic interaction between a user and virtual characters. For example, the systems and methods describe techniques for mathematically generating a map used for animating facial expressions in a multidimensional animation blendspace. As another example, the systems and methods describe a transition system for dynamically transitioning facial expressions across a face of the virtual character. As another example, realistic physical movements can be added to a virtual character's facial expressions to provide interactivity with other virtual characters.

Avatar facial expression representation in multidimensional space

Real-time animation of a virtual character can be provided by pre-calculating a control policy that identifies a suitable animation clip to animate the next movement of the virtual character. The control policy can be calculated using a Markov decision process (MDP) and can specify an action to take (e.g., a movement) when the virtual character is in a particular state (e.g., in a particular position or pose). The control policy can be determined based on a discounted set of rewards or punishments associated with various actions that can occur in each state. The control policies can be pre-calculated, offline from a runtime animation engine (e.g., in an augmented reality display). The runtime animation engine can use the control policies to select an animation clip to move the virtual character through an environment in real time.

Avatar animation using markov decision process policies

Examples of systems and methods for rendering an avatar in a mixed reality environment are disclosed. The systems and methods may be configured to automatically select avatar characteristics that optimize gaze perception by the user, based on context parameters associated with the virtual environment.

Avatar customization for optimal gaze discrimination

Avatar animation using Markov decision process policies

Growing community of inventors

Los Angeles, CA, United States of America

Thomas Marshall Miller, Iv