Apple Inc.

Daniel Laszlo Kovacs

Payal Jotwani

Mark E Drummond

Siva Chandra Mouli Sivapurapu

Michael J Gutensohn

Dan Feng

Bo Morgan

Edward S Ahn

Aashi Manglik

Mu Qiao

Shaun D Budhram

Behrooz Mahasseni

Noah Jonathan Gamboa

Michelle Chua

Bradley Warren Peebler

Adam Michael O'Hern

Noah Gamboa

James Moll

A device obtains a request to display a graphical environment. The graphical environment is associated with a set of saliency values corresponding to respective portions of the graphical environment. A POV for displaying the graphical environment is selected based on the set of saliency values. The graphical environment is displayed from the selected POV on the display.

Displaying an environment from a selected point-of-view

In one implementation, a method of composing a scene content is performed at a device including a display, one or more processors, and non-transitory memory. The method includes generating a definition of a scene based on textual or speech input and a model of a physical environment, wherein the definition includes a constraint that defines a spatial relationship between a virtual asset and an anchor asset that corresponds to one or more physical objects in the physical environment. The method includes generating, based on the definition of the scene and the model of the physical environment, a first instance of the scene that satisfies the constraint with the virtual asset in the spatial relationship with a first one of the one or more physical objects in the physical environment. The method includes presenting, on the display, the first instance of the scene.

Methods and systems for composing and executing a scene

Various implementations disclosed herein include a method performed by an electronic device including one or more processors and a non-transitory memory. The method includes obtaining first environmental data that characterizes a first physical setting that is spatially distinct from a second physical setting. The second physical setting includes the electronic device. The method includes generating, using the first environmental data, a first extended reality (XR) environment corresponding to the first physical setting. The method includes generating a first mapping between a first bounding surface of the second physical setting and a first bounding surface of the first XR environment.

Connecting spatially distinct settings

A method includes obtaining, by a first agent engine that generates actions for a first agent, a first objective of the first agent. In some implementations, the method includes generating, by the first agent engine, a first influence for a second agent engine that generates actions for a computer-generated reality (CGR) representation of a second agent. In some implementations, the first influence is based on the first objective of the first agent. In some implementations, the method includes triggering the CGR representation of the second agent to perform a set of one or more actions that advances the first objective of the first agent. In some implementations, the second agent engine generates the set of one or more actions based on the first influence generated by the first agent engine.

Influencing actions of agents

In one implementation, a method for recording an XR environment. The method includes: presenting, via the display device, a graphical environment with one or more virtual agents, wherein the graphical environment corresponds to a composition of extended reality (XR) content, including the one or more virtual agents, and an image stream of a physical environment captured from a first point-of-view (POV) of the physical environment; detecting, via the one or more input devices, a user input selecting a first virtual agent from among the one or more virtual agents; and in response to detecting the user input, recording a plurality of data streams associated with the graphical environment including a first image stream of the graphical environment from the first POV and one or more data streams of the graphical environment from a current POV of the first virtual agent.

Method and device for debugging program execution and content playback

In one implementation, a method of displaying content is performed at a device including a display, one or more processors, and non-transitory memory. The method includes scanning a first physical environment to detect a first physical object in the first physical environment and a second physical object in the first physical environment, wherein the first physical object meets at least one first object criterion and the second physical object meets at least one second object criterion. The method includes displaying, in association with the first physical environment, a virtual object moving along a first path from the first physical object to the second physical object. The method includes scanning a second physical environment to detect a third physical object in the second physical environment and a fourth physical object in the second physical environment, wherein the third physical object meets the at least one first object criterion and the fourth physical object meets the at least one second object criterion. The method includes displaying, in association with the second physical environment, the virtual object moving along a second path from the third physical object to the fourth physical object, wherein the second path is different than the first path.

In some implementations, a method of navigation mesh exploration is performed at a virtual agent operating system. The method includes: determining one or more first sensory perception regions for one or more senses of a virtual agent based on a first perceptual vector associated with the virtual agent; generating a first portion of a navigation mesh for the XR environment based on the one or more first sensory perception regions, wherein the first portion of the navigation mesh includes candidate subsequent locations different from the first location; and in response to detecting movement of the virtual agent to a respective candidate subsequent location among candidate subsequent locations, generating a second portion of the navigation mesh for the XR environment based on one or more second sensory perception regions for the one or more senses of the virtual agent relative to the respective candidate subsequent location.

Method and device for navigation mesh exploration

In one implementation, a method for improved motion planning. The method includes: obtaining a macro task for a virtual agent within a virtual environment; generating a search-tree based on at least one of the macro task, a state of the virtual environment, and a state of the virtual agent, wherein the search-tree includes a plurality of task nodes corresponding to potential tasks for performance by the virtual agent in furtherance of the macro task; and determining physical motion plans (PMPs) for at least some of the plurality of task nodes within the search-tree in order to generate a lookahead planning gradient for the first time, wherein a granularity of a PMP for a respective task node in the first search-tree is a function of the temporal distance of the respective task node from the first time.

Method and device for improved motion planning

Various implementations disclosed herein include devices, systems, and methods for controlling motion of CGR objects. In various implementations, a device includes a non-transitory memory and one or more processors coupled with the non-transitory memory. In some implementations, a method includes determining, by a first animation controller, values for a first set of animation parameters associated with a first animation for a computer-generated reality (CGR) object. In some implementations, the CGR object is associated with a plurality of joints. In some implementations, the method includes generating, by a motion controller, respective joint movements for the plurality of joints based on the values for the first set of animation parameters. In some implementations, the method includes manipulating the CGR object in accordance with the respective joint movements for the plurality of joints in order to provide an appearance that CGR object is moving within a degree of similarity to the first animation.

Motion control for an object

In some implementations, a method of improved pathfinding is performed at a virtual agent operating system including non-transitory memory and one or more processors coupled with the non-transitory memory. The method includes: determining an initial path for a virtual agent to a target destination based at least in part on a navigation mesh of an XR environment; actuating locomotive elements of the virtual agent in order to move the virtual agent according to the initial path; while moving according to the initial path, detecting a node of a navigation graph; in response to detecting the node of the navigation graph: obtaining navigation information from the node of the navigation graph; and determining an updated path from the node to the target destination based at least in part on the navigation mesh and the navigation information; and actuating the locomotive elements of the virtual agent according to the updated path.

Method and device for improved pathfinding

A method includes maintaining a hierarchical datastore for an agent instantiated in a computer-generated reality (CGR) environment. The hierarchical datastore includes a first storage hierarchy associated with a first data type and a second storage hierarchy associated with a second data type. The method includes detecting a sensory input that includes sensory input data. The method includes determining a type of the sensory input data. The method includes, in response to the sensory input data being of the first data type, storing the sensory input data in the first storage hierarchy for a first amount of time associated with the first storage hierarchy. The method includes, in response to the sensory input data being of the second data type, storing the sensory input data in the second storage hierarchy for a second amount of time that is different from the first amount of time.

Growing community of inventors

Santa Clara, CA, United States of America

Daniel Laszlo Kovacs