Apple Inc.

Jean-Charles Bernard Marcel Bazin

Anselm Grundhoefer

Arun Srivatsan Rangaprasad

Thomas J Moore

Ivan Kolesov

Bart Colin Trzynadlowski

Mohamed Selim Ben Himane

Shubham Agrawal

Joseph M Luxton

Nathaniel R Finney

Dhruv A Govil

Srimanth Gunturi

Various implementations disclosed herein include devices, systems, and methods that provide point identification techniques for electronic devices such as optical see-through head mounted devices. In some implementations, a line of sight technique is used to identify a 3D position of a point. In some implementations, a touching technique is used to identify a 3D position of a point. In some implementations, different point identification techniques are automatically selected and used to identify a 3D position of a point. In some implementations, a 3D position of a point is associated with user input. In some implementations, a 3D position of a point is identified to determine distances, surface areas, or volumes.

Three-dimensional point selection

Hand tracking techniques are used to initiate actions at a computer even if a hand is occupied. Image data is received of a scene that includes a hand. A determination is mate whether the hand is occupied by a physical object. If the hand is occupied by a physical object, then the hand is monitored for a modified gesture based on the physical object. Some objects may be pre-registered and associated with object-specific actions for various gestures. When a user is within a predetermined distance of a registered object, a system monitors a user's hands for a gesture, and then triggers an action based on the gesture and the object.

Determining an intended gesture when hands are occupied

Various implementations disclosed herein include devices, systems, and methods that provide XR in which virtual objects are positioned based on the accuracy of localizing an electronic device in a physical environment. In some implementations, the technique assesses the accuracy of localization (e.g., centimeter-level accuracy, room-level accuracy, and building-level accuracy) and dynamically adjusts a display strategy. In some implementations, the technique determines a condition causing inaccuracy (e.g., a semantic condition such as 'too fast', “too far”, “too dark”), and provides a notification (e.g., “too fast-slow down”, “too far-move closer”, “too dark-turn on a light”) at the electronic device based on the condition causing the inaccuracy in the localization.

Localization accuracy response

Various implementations disclosed herein include devices, systems, and methods that provide measurement techniques for electronic devices such as optical see-through head mounted devices. In some implementations, a line of sight technique is used to identify a 3D position of a measurement point to enable measurement of an object in a 3D environment. In some implementations, different measurement point identification techniques are automatically selected and used to identify a 3D position of a measurement point to enable measurement of an object in a 3D environment. In some implementations, a 3D position of a measurement point is identified to enable measurement of an object in a 3D environment, where the measurement point is identified by selecting from multiple candidates that are determined using different measurement point selection techniques.

Measurement based on point selection

Various implementations disclosed herein include devices, systems, and methods that provide a visual marker including a plurality of markings arranged in a corresponding plurality of shapes. In some implementations, each marking is formed of a set of sub-markings separated by gaps and arranged according to a respective shape, and the gaps of the plurality of markings are configured to encode data and indicate orientation of the visual marker. In some implementations, the plurality of markings are arranged in a plurality of concentric rings of increasing size. In some implementations, the orientation is encoded in a first set of gaps and data in a second set of gaps of the gaps in the plurality of markings.

Visual marker

Various implementations disclosed herein include devices, systems, and methods that match image portions of a first set of image portions and image portions of another set of image portions. This may be performed in a such a way that the sum of descriptor distances of the matched image portions is optimized in a globally optimal way. In some implementations, the matching is subject to one or more constraints, e.g., ensuring that each image portion of the first data set is matched with at least one image portion of the second data set and ensuring that each image portion of the second data set is matched with only one image portion of the first data set.

Optimal matching of image portion data sets

Various implementations disclosed herein include devices, systems, and methods that provide XR in which virtual objects are positioned based on the accuracy of localizing an electronic device in a physical environment. In some implementations, the technique assesses the accuracy of localization (e.g., centimeter-level accuracy, room-level accuracy, and building-level accuracy) and dynamically adjusts a display strategy. In some implementations, the technique determines a condition causing inaccuracy (e.g., a semantic condition such as 'too fast', 'too far', 'too dark'), and provides a notification (e.g., &#x201c;too fast-slow down&#x201d;, &#x201c;too far-move closer&#x201d;, &#x201c;too dark-turn on a light&#x201d;) at the electronic device based on the condition causing the inaccuracy in the localization.

Growing community of inventors

Sunnyvale, CA, United States of America

Jean-Charles Bernard Marcel Bazin