Apple Inc.

Chase B Lortie

David J Meyer

Julian K Shutzberg

Daniel J Brewer

Ali Ercan

David M Teitelbaum

Lorenzo Soto Doblado

Wei-Kang Hsu

Various implementations assess user interactions to trigger user interface responses. Some implementations associate user gaze with user interface elements using one or more criteria that enable interpreting the user's gaze in ways that are consistent with user intentions and/or expectations. In some implementations, a user's gaze is associated with a user interface element when the user gazes at the user interface element. When the gaze leaves the user interface element (e.g., intentionally or unintentionally), the association may be maintained or discontinued using one or more criteria to be consistent with user intentions and/or expectations. In some implementations, this involves determining whether to continue associating a user's gaze with a user interface element so that the gaze-to-element association is maintained in certain circumstances (e.g., blinks and saccades) but discontinued in other circumstances (e.g., when the user is fixating outside of a zone around the element for more than a threshold time/number of frames).

Optimizations in associating user gaze with user interface elements

Various implementations disclosed interpret direct touch-based gestures, such as drag and scroll gestures, made by a user virtually touching one position of a user interface and moving their hand to another position of the user interface. For example, such gestures may be made relative to a user interface presented in an extended reality (XR) environment. In some implementations, a user movement is interpreted using one or more techniques that avoid unexpected gain or loss of user-interface-associated motion. Some implementations determine which segments of a movement to associate with user interface content based on characteristics of the movement. Some implementations determine that a break occurs when a user movement leaves a break volume that is adjusted dynamically.

Interpreting user movement as direct touch user interface interactions

Various implementations disclosed herein include devices, systems, and methods that associate user activities with user interface (UI) elements within a 3D environment. Some implementations do so in ways that account for inaccuracies (i.e., fuzziness) in sensor-based detection of the user activities, e.g., inaccuracy in sensor data-based gaze tracking or sensor data-based hand/joint position. Some implementations use a sampling technique to associate user activity in a 3D environment with an appropriate portion of a UI positioned within a 3D environment. For example, a sampling technique may be used to identify sample locations within the 3D environment (e.g., sample locations around a gaze direction) to evaluate and ultimately select from to associate with a user activity, e.g., associating a particular UI button with a gaze direction.

Fuzzy hit testing

Growing community of inventors

San Francisco, CA, United States of America

Chase B Lortie