Meta Platforms Technologies, LLC

Microsoft Technology Licensing, LLC

Facebook Technologies, LLC

Other

Michal Hlavac

Arthur Zwiegincew

Barrett Fox

Hayden Schoen

James Tichenor

Gregory Alt

Todd K Harris

Merlyn Deng

Alex Marcolina

Alexander Michael Louie

Pol Pla I Conesa

Jasper Stevens

Wai Leong Chak

Yeliz Karadayi

Gioacchino Noris

Anush Mohan

Christopher Richard Tanner

Paul Timothy Furgale

Jossie E Tirado Arroyo

Johannes Joachim Schmid

Benjamin Taylor Koga-Winn

Mark D Schwesinger

Brian Murphy

Bernhard Poess

David M Hill

Jia Wang

Ryan James D'Aurelio

Jenny Kam

Emiko Valiane Charbonneau

John Jacob Blakeley

Michael Ishigaki

Ana Sofia Gonzalez Aguirre

Travis William Steiner

Mark A DeFalco

Jonathan Michael Proto

Paul Mealy

Kevin Harper

Benjamin Taylor Winn

Michael T Hochkeppel

Charles Lapp

Arthur Ramazanov

Pedro Ramirez

Shannon Richard Monroe

Ryan J D'AURELIO

An artificial reality environment (XRE) schema is defined that supports controlling interactions between various artificial reality actors. The XRE schema includes a set of definitions for an XRE, independent of type of artificial reality device. The definitions in the XRE schema can include standards for both interfaces and data objects. The XRE schema can define XR elements in terms of entities and components of a space, organized according to a hierarchy. Each entity can represent a real or virtual object or space, within the XRE, defined by a name and a collection of one or more components. Each component (as part of an entity) can define aspects and expose information about the entity. The XRE schema can specify structures that that allow actors (e.g., producers, instantiators, and consumers) to define and perform actions in relation to XRE elements.

Artificial reality environment control through an artificial reality environment schema

Aspects of the present disclosure are directed to providing an artificial reality environment with augments and surfaces. An 'augment' is a virtual container in 3D space that can include presentation data, context, and logic. An artificial reality system can use augments as the fundamental building block for displaying 2D and 3D models in the artificial reality environment. For example, augments can represent people, places, and things in an artificial reality environment and can respond to a context such as a current display mode, time of day, a type of surface the augment is on, a relationship to other augments, etc. Augments can be on a “surface” that has a layout and properties that cause augments on that surface to display in different ways. Augments and other objects (real or virtual) can also interact, where these interactions can be controlled by rules for the objects evaluated based on information from the shell.

Artificial reality augments and surfaces

In one embodiment, a method includes instructing sensors on an AR headset to capture data of a real-world environment, discovering a 3D location where a native application configured to run natively on an operating system is downloadable, generating and displaying a virtual object representing the native application at the 3D location, detecting a user of the AR system interacting with the virtual object, downloading a first executable portion of the native application, executing the first executable portion within a sandbox process having a predetermined set of access permissions, progressively downloading a remaining executable portion of the native application while the user is interacting with the executed first executable portion, and in response to a determination that the native application is completely downloaded, installing the native application on the operating system.

Ephemeral artificial reality experiences

A virtual object system can orchestrate virtual objects defined as a collection of components and with inheritance in an object hierarchy. Virtual object components can include a container, data, a template, and a controller. A container can define the volume the virtual object is authorized to write into. A virtual object's data can specify features such as visual elements, parameters, links to external data, meta-data, etc. The template can define view states of the virtual object and contextual breakpoints for transitioning between them. Each view state can control when and how the virtual object presents data elements. The controller can define logic for the virtual object to respond to input, context, etc. The definition of each object can specify which other object in an object hierarchy that object extends, where extending an object includes inheriting that object's components, which can be modified or overwritten as part of the extension.

Virtual object structures and interrelationships

The present disclosure provides world-controlled augments and application-controlled augments. World-controlled augments can be controlled directly by a shell in the artificial reality environment. To allow even inexperienced users to develop world-controlled augments, a world-controlled builder system is provided. Application-controlled augments may be resource intensive (e.g., using eye-tracking, social-media tie-ins, etc.), may support complicated interactions among themselves, may require or have extensive use of inputs and permissioned resources, and are controlled by their hosting application. When a running application is halted, the application closes its application-controlled augments but can choose to have the XR system run a 'place-holder' world-controlled augment for as long as the application is not running. The place-holder world-controlled augment preserves the appearance of the application in the artificial reality environment but uses few system resources. Upon a user selecting a place-holder world-controlled augment, the XR system can run the associated application and close the place-holder world-controlled augment.

World-controlled and application-controlled augments in an artificial-reality environment

An artificial reality environment (XRE) schema is defined that supports anel-controlling interactions between various artificial reality actors. The XRE schema includes a set of definitions for an XRE, independent of type of artificial reality device. The definitions in the XRE schema can include standards for both interfaces and data objects. The XRE schema can define XR elements in terms of entities and components of a space, organized according to a hierarchy. Each entity can represent a real or virtual object or space, within the XRE, defined by a name and a collection of one or more components. Each component (as part of an entity) can define aspects and expose information about the entity. The XRE schema can specify structures that that allow actors (e.g., producers, instantiators, and consumers) to define and perform actions in relation to XRE elements.

In one embodiment, a method includes, by an operating system of a first artificial-reality device, receiving a notification that virtual objects are shared with the first artificial-reality device by a second artificial-reality device, where the virtual objects are shared by being placed inside a sender-side shared space anchored to a physical object. The method further includes the first artificial-reality device accessing descriptors of a physical object and a spatial-relationship definition between the physical object and a receiver-side shared space, detecting physical objects based on the descriptors, determining pose of the receiver-side shared space, detecting physical constraints within the receiver-side shares space, receiving display instructions for the virtual objects, and rendering the virtual objects on the first artificial-reality device in the receiver-side shared space.

Systems and methods for content sharing between artificial-reality devices

Aspects of the present disclosure are directed to an artificial reality system orchestrating interactions between virtual object 'augments.' The orchestration can include linking, which can be forming two or more augments into a combination, embedding an augment within an existing combination, or triggering an action mapped to the linking of those augments. Another type of orchestration can include extracting, which can refer to taking an augment out of an existing combination, either by removing it from the combination or copying the augment to leave a version in the combination and having another version outside the combination.

Augment orchestration in an artificial reality environment

Aspects of the present disclosure are directed to providing an artificial reality environment with augments and surfaces. An 'augment' is a virtual container in 3D space that can include presentation data, context, and logic. An artificial reality system can use augments as the fundamental building block for displaying 2D and 3D models in the artificial reality environment. For example, augments can represent people, places, and things in an artificial reality environment and can respond to a context such as a current display mode, time of day, a type of surface the augment is on, a relationship to other augments, etc. Augments can be on a 'surface' that has a layout and properties that cause augments on that surface to display in different ways. Augments and other objects (real or virtual) can also interact, where these interactions can be controlled by rules for the objects evaluated based on information from the shell.

Aspects of the present disclosure are directed to an artificial reality (XR) application system controlling applications in an artificial reality environment. In various cases, these controls include automatically suggesting XR applications by determining an XR context and identifying applications that match the XR context. These applications can be suggested to a user, who can authorize their execution, setting permissions for the application. In some cases, applications can be divided into components which can be progressively downloaded. By providing application suggestions relevant to the current context and progressively downloading application components, applications can appear ambient, rather than relying on users to constantly download, install, or activate applications. Permissions for applications may be revoked permanently or for certain situations&#x2014;either through user permissions selections or automatically in response to determined user intents. When multiple applications are simultaneously authorized to execute, the XR application system can employ a ranking system to prevent overcrowding.

Artificial reality application lifecycle

Artificial realty augments and surfaces

Growing community of inventors

Seattle, WA, United States of America

Michal Hlavac