Magic Leap, Inc.

Apple Inc.

Victor Ng-Thow-Hing

Tomislav Pejsa

Brian Everett Meaney

David Charles Lundmark

Shivakumar Balasubramanyam

Marc Coleman Shelton

Megan Ann Geiman

John A Gotcher

Matthew Schon Bogue

Jeffrey Edward Ruediger

Leonid Zolotarev

Benjamin Zaaron Reinhardt

Aleksei Marchenko

Denys Bastov

Yannick Pellet

Lorena Pazmino

Karen Stolzenberg

Gregory Minh Tran

Cole Parker Heiner

Thomas Marshall Miller, Iv

Frank Alexander Hamilton, Iv

Rodrigo Cano

Federico Schliemann

Josh Anon

Stephane Antoine Joseph Imbert

Anthony Marinello

Francisco Lopez-Fresquet

Koichi Mori

Dushan Vasilevski

Frank Alexander Hamilton Iv

Thomas Marshall Miller Iv

Systems and methods for matching content elements to surfaces in a spatially organized 3D environment. The method includes receiving content, identifying one or more elements in the content, determining one or more surfaces, matching the one or more elements to the one or more surfaces, and displaying the one or more elements as virtual content onto the one or more surfaces.

Matching content to a spatial 3D environment

Examples of systems and methods for rendering an avatar in a mixed reality environment are disclosed. The systems and methods may be configured to automatically scale an avatar or to render an avatar based on a determined intention of a user, an interesting impulse, environmental stimuli, or user saccade points. The disclosed systems and methods may apply discomfort curves when rendering an avatar. The disclosed systems and methods may provide a more realistic interaction between a human user and an avatar.

Contextual-based rendering of virtual avatars

Various implementations disclosed herein include devices, systems, and methods that provide a view of an adjusted position of a three-dimensional (3D) representation of another user (e.g., an avatar) during a communication session. At a first device, 3D representation data corresponding to at least a portion of a second user of a second device (e.g., a sender) is obtained during a communication session. Offset information (e.g., a vertical offset value) corresponding to an offset between a camera viewpoint position and an eye viewpoint position associated with the second device is obtained. A position of a first portion of a 3D representation of the second user (e.g., head and torso) is adjusted relative to a 3D environment associated with the first device. A view that includes the first portion of the 3D representation of the second user in the 3D environment is provided during the communication session.

3D representation adjustments for communication sessions

Wearable systems for privacy preserving expression generation for augmented or virtual reality client applications. An example method includes receiving, by an expression manager configured to communicate expression information to client applications, a request from a client application for access to the expression information. The expression information reflects information derived from one or more sensors of the wearable system, with the client application being configured to present virtual content including an avatar rendered based on the expression information. A user interface is output for presentation which requests user authorization for the client application to access the expression information. In response to receiving user input indicating user authorization, enabling access to the expression information is enabled. The client application obtains periodic updates to the expression information, and the avatar is rendered based on the periodic updates.

Privacy preserving expression generation for augmented or virtual reality client applications

Examples of systems and methods for improved hand tracking of a user in a mixed reality environment are disclosed. The systems and methods may be configured to estimate the hand pose and shape of a user's hands for applications such as animating a hand on a user's avatar. Data from multiple sources, such as a totem internal measurement unit ('IMU'), external totem location tracking, vision cameras, and depth sensors, may be manipulated using a set of rules that are based on historical data, ergonomics data, and motion data.

Multi-modal hand location and orientation for avatar movement

A method of generating a simulation of a physical object in an image of a physical environment. The method also includes generating a planar polygon mesh from at least the image. The method further includes extracting a boundary polygon of the planar polygon mesh. Moreover, the method includes generating a convex hull for the boundary polygon of the surface mesh. In addition, the method includes generating a minimal area oriented boundary polygon from the convex hull. The method may also include generating a maximal area oriented internal polygon (MAOBP) inside of the boundary polygon of the planar polygon mesh. The MAOBP is utilized to generate a 3-D surface model of the physical object, and the 3-D surface model is used to generate a simulation involving an interaction with the 3-D surface model of the physical object.

Surface modeling systems and methods

An augmented reality viewer is described. A user orientation determination module determines a user orientation. A content vector calculator calculates a content orientation vector relative to a near edge and a far edge of content, determines a dot product of the user orientation vector and the content orientation vector, and positions the content based on a magnitude of the dot product. A surface area vector calculator calculates a surface area orientation vector for each of a plurality of surface area. A surface selection module determines a dot product of the user orientation vector and each surface area orientation vector and selects a preferred surface based on the relative magnitude of the dot products.

Augmented reality viewer with automated surface selection placement and content orientation placement

A method of generating a surface model of a physical environment includes obtaining an image of the physical environment. The method also includes generating a planar polygon mesh from at least the image. The method further includes extracting a boundary polygon of the planar polygon mesh. Moreover, the method includes generating a convex hull for the boundary polygon of the surface mesh. In addition, the method includes generating a minimal area oriented boundary polygon from the convex hull. The method may also include generating a maximal area oriented internal polygon inside of the boundary polygon of the planar polygon mesh.

Growing community of inventors

Los Altos, CA, United States of America

Victor Ng-Thow-Hing