Apple Inc.

Bruno M Sommer

Connor A Smith

Alexandre Da Veiga

Timothy Robert Oriol

Jeffrey S Norris

Aaron Mackay Burns

Tucker Bull Morgan

Norman Nuo Wang

Ross R Dexter

Shih Sang Chiu

Miao Ren

Aurelio Guzmán

Jonathan R Dascola

Avi Bar-Zeev

Nicolas V Scapel

Hoan K Pham

Rahul Nair

Guillaume P Barlier

Thomas Goossens

Alexis Henri Palangie

Jacques P Gasselin De Richebourg

Stephen E Pinto

Alexandre R Moha

Justin D Stoyles

Nicholas W Henderson

Shaun D Budhram

Frank Angermann

Clément Pierre Nicolas Boissière

Thibaut Weise

Brian Amberg

Lilli Ing-Marie Jonsson

Nina Damasky

Timofey Grechkin

Moinul H Khan

Hao Pan

Berkat S Tung

Geoffrey Grant Stahl

Nicolas P Bonnier

Tobias Eble

Luis R Deliz Centeno

Srinivas Vedula

Joel N Kerr

Ioana Negoita

James J Cwik

Ye Cong

Michael Ryan Brennan

Blerim Cici

Hayden J Lee

Leanid Vouk

Facilitating collaboration in a multi-user communication session may include, at a first device, providing a toolkit of system-level tools for interacting within the session with a second device. A first user interaction with a first tool is detected which results in a virtual modification to an environment of the first device. In response to the first user interaction, the virtual modification is provided for presentation in a second environment representation of the session corresponding to the second device.

Toolbox and context for user interactions

In one implementation, a method of resolving focal conflict in a computer-generated reality (CGR) environment is performed by a device including a processor, non-transitory memory, an image sensor, and a display. The method includes capturing, using the image sensor, an image of a scene including a real object in a particular direction at a first distance from the device. The method includes displaying, on the display, a CGR environment including a virtual object in the particular direction at a second distance from the device. In accordance with a determination that the second distance is less than the first distance, the CGR environment includes the virtual object overlaid on the scene. In accordance with a determination that the second distance is greater than the first distance, the CGR environment includes the virtual object with an obfuscation area that obfuscates at least a portion of the real object within the obfuscation area.

Method and device for resolving focal conflict

Avatars may be displayed in a multiuser communication session using various spatial modes. One technique for presenting avatars includes presenting avatars such that an attention direction of the avatar is retargeted to match the intent of the remote user corresponding to the avatar. Another technique for presenting avatars includes a pinned mode in which a spatial relationship between one or more avatars remains displayed in a consistent spatial relationship to a local user regardless of movements of the local user. Another technique for presenting avatars includes providing user-selectable presentation modes between a room scale mode and a stationary mode for presenting a representation of a multiuser communication session.

Avatar spatial modes

Various implementations disclosed herein include devices, systems, and methods for presenting communication data. In various implementations, a first device associated with a first person includes a display, a non-transitory memory and one or more processors coupled with the display and the non-transitory memory. In some implementations, a method includes obtaining communication data associated with a second device corresponding to a second person. In some implementations, the method includes determining whether the first device and the second device are in a shared environment. In some implementations, the method includes in response to determining that the first device and the second device are not in a shared environment, displaying a representation of the second person based on the communication data associated with the second device.

Presenting communication data based on environment

Techniques for displaying a virtual object in an enhanced reality setting in accordance with a physical muting mode being active are described. In some examples, while the physical muting mode is active, a system obtains a first set of context data for one or more physical elements in a physical setting and causes display of a virtual object representing the one or more physical elements in a first position using at least a portion of the first set of context data. After the virtual object representing the one or more physical elements is displayed in the first position, the physical muting mode is deactivated. In response to detecting the deactivation of the physical muting mode, the system causes display of a transitional motion of the virtual object representing the one or more physical elements from the first position to a second position.

Muting mode for a virtual object representing one or more physical elements

Some examples of the disclosure are directed to selective display of avatars corresponding to users of electronic devices in a multi-user communication session. In some examples, when immersive content is shared in the communication session, the avatars remain displayed when presenting the content in the three-dimensional environment. In some examples, when perspective-limited immersive content is shared in the communication session, the avatars cease being displayed when presenting the content in the three-dimensional environment. In some examples, when content presented in a full-screen mode is shared in the communication session, the avatars remain displayed when presenting the content in the full-screen mode in the three-dimensional environment. In some examples, when object-bounded content is shared in the communication session, the avatars remain displayed when presenting the object-bounded content in the three-dimensional environment.

System and method of three-dimensional immersive applications in multi-user communication sessions

Various implementations disclosed herein include devices, systems, and methods that enable a device to provide a view of virtual elements and a physical environment where the presentation of the virtual elements is based on positioning relative to the physical environment. In one example, a device is configured to detect a change in positioning of a virtual element, for example, when a virtual element is added, moved, or the physical environment around the virtual element is changed. The location of the virtual element in the physical environment is used to detect an attribute of the physical environment upon which the presentation of the virtual element depends. Thus, the device is further configured to detect an attribute (e.g., surface, table, mid-air, etc.) of the physical environment based on the placement of the virtual element and present the virtual element based on the detected attribute.

Location-based virtual element modality in three-dimensional content

Methods and apparatus for spatial audio navigation that may, for example, be implemented by mobile multipurpose devices. A spatial audio navigation system provides navigational information in audio form to direct users to target locations. The system uses directionality of audio played through a binaural audio device to provide navigational cues to the user. A current location, target location, and map information may be input to pathfinding algorithms to determine a real world path between the user's current location and the target location. The system may then use directional audio played through a headset to guide the user on the path from the current location to the target location. The system may implement one or more of several different spatial audio navigation methods to direct a user when following a path using spatial audio-based cues.

Spatial audio navigation

A method includes receiving one or more signals that each indicate a device type for a respective remote device, identifying one or more visible devices in one or more images, matching a first device from the one or more visible devices with a first signal from the one or more signals based on a device type of the first device matching a device type for the first signal and based on a visible output of the first device, pairing the first device with a second device, and controlling a function of the first device using the second device.

Discovery of and connection to remote devices

A device for providing operating system managed group communication sessions may include a memory and at least one processor. The at least one processor may be configured to receive, by an operating system level process executing on a device and from an application process executing on a device, a request to initiate a group session between a user associated with the device and another user. The at least one processor may be further configured to identify, by the operating system level process, another device associated with the other user. The at least one processor may be further configured to initiate, by the operating system level process, the group session with the user via the other device. The at least one processor may be further configured to manage, by the operating system level process, the group session.

Operating system level management of group communication sessions

Techniques for displaying a virtual object in an enhanced reality setting in accordance with a physical muting mode being active are described. In some examples, a system obtains context data for one or more physical elements in a physical setting, wherein the context data includes first context data and second context data that is different from the first context data. In some examples, in response to obtaining the context data for the one or more physical elements in the physical setting, a system causes display of a virtual object that represents the one or more physical elements using the first context data without using the second context data, in accordance with a determination that a physical muting mode is active.

A method may include receiving a communication from a device at an artificial intelligence controller including state information for a software application component running on the device, the state information including information corresponding to at least one potential state change available to the software application component, and metrics associated with at least one end condition, interpreting the state information using the artificial intelligence controller, and selecting an artificial intelligence algorithm from a plurality of artificial intelligence algorithms for use by the software application component based on the interpreted state information; and transmitting, to the device, an artificial intelligence algorithm communication, the artificial intelligence algorithm communication indicating the selected artificial intelligence algorithm for use in the software application component on the device.

Artificial intelligence controller that procedurally tailors itself to an application

A method includes obtaining first pass-through image data characterized by a first pose. The method includes obtaining respective pixel characterization vectors for pixels in the first pass-through image data. The method includes identifying a feature of an object within the first pass-through image data in accordance with a determination that pixel characterization vectors for the feature satisfy a feature confidence threshold. The method includes displaying the first pass-through image data and an AR display marker that corresponds to the feature. The method includes obtaining second pass-through image data characterized by a second pose. The method includes transforming the AR display marker to a position associated with the second pose in order to track the feature. The method includes displaying the second pass-through image data and maintaining display of the AR display marker that corresponds to the feature of the object based on the transformation.

Methods and devices for detecting and identifying features in an AR/VR scene

Growing community of inventors

Sunnyvale, CA, United States of America

Bruno M Sommer