Apple Inc.

Daniel J Brewer

Julian K Shutzberg

David J Meyer

Victor Belyaev

Christopher D Mckenzie

Leah M Gum

Ashwin Kumar Asoka Kumar Shenoi

Chase B Lortie

David M Teitelbaum

Yirong Tang

Tian Qiu

Bhavin Vinodkumar Nayak

Itay Bar Yosef

Matthias M Schroeder

Bharat C Dandu

Suppressing a hand gesture upon detecting peripheral events on a peripheral device includes determining a first hand pose for a first hand a second hand pose for a second hand in response to a detected peripheral device peripheral event, determining, based on the first hand pose and the second hand pose, at least one hand of the first hand and the second hand in a peripheral use mode, detecting an input gesture from a hand of the at least one hand determined to be in the peripheral use mode, and rejecting the input gesture by a user input pipeline in accordance with the determination that the hand is in the peripheral use mode. The presence of a peripheral device is confirmed by activating a computer vision system in response to determining that a peripheral use condition is satisfied.

Suppression of hand gestures upon detection of peripheral events on a peripheral device

Various implementations disclosed herein include devices, systems, and methods that interpret user activity as user interactions with user interface (UI) elements positioned within a three-dimensional (3D) space such as an extended reality (XR) environment. Some implementations enable user interactions with virtual elements displayed in 3D environments that utilize alternative input modalities, e.g., XR environments that interpret user activity as either direct interactions or indirect interactions with virtual elements.

Input recognition based on distinguishing direct and indirect user interactions

Various implementations disclosed interpret direct touch-based gestures, such as drag and scroll gestures, made by a user virtually touching one position of a user interface and moving their hand to another position of the user interface. For example, such gestures may be made relative to a user interface presented in an extended reality (XR) environment. In some implementations, a user movement is interpreted using one or more techniques that avoid unexpected gain or loss of user-interface-associated motion. Some implementations determine which segments of a movement to associate with user interface content based on characteristics of the movement. Some implementations determine that a break occurs when a user movement leaves a break volume that is adjusted dynamically.

Interpreting user movement as direct touch user interface interactions

Enabling gesture input includes obtaining hand tracking data based on one or more camera frames, detecting a contact event between a first finger and a second finger based on the hand tracking data, and determining a first contact location on the first finger and a second contact location on the second finger. In accordance with a determination that the first contact location and the second contact location are within a first predefined gesture zone for a first gesture, an input action is enabled corresponding to the first gesture.

Pinch recognition using finger zones

Growing community of inventors

San Jose, CA, United States of America

Daniel J Brewer