Apple Inc.

Daniel J Brewer

Julian K Shutzberg

David J Meyer

Leah M Gum

Chase B Lortie

Yirong Tang

Victor Belyaev

Christopher D Mckenzie

Ashwin Kumar Asoka Kumar Shenoi

David M Teitelbaum

Itay Bar Yosef

Alexander T Wing

Tian Qiu

Bhavin Vinodkumar Nayak

Matthias M Schroeder

Bharat C Dandu

A movement of a user input component is presented in accordance with a mapping of a user input motion from a user-centric spherical coordinate system to a 2D plane. An input motion by a user is detected. An origin for the input motion is identified in a user-centric spherical coordinate system. An arc length for the input motion is determined based on the identified origin. The input motion is mapped from the user-centric spherical coordinate system to a 2D plane of a user input component, and a movement of the user input component is presented on the 2D plane in accordance with the mapping.

Motion mapping for continuous gestures using user-centric coordinate system

Suppressing a hand gesture upon detecting peripheral events on a peripheral device includes determining a first hand pose for a first hand a second hand pose for a second hand in response to a detected peripheral device peripheral event, determining, based on the first hand pose and the second hand pose, at least one hand of the first hand and the second hand in a peripheral use mode, detecting an input gesture from a hand of the at least one hand determined to be in the peripheral use mode, and rejecting the input gesture by a user input pipeline in accordance with the determination that the hand is in the peripheral use mode. The presence of a peripheral device is confirmed by activating a computer vision system in response to determining that a peripheral use condition is satisfied.

Suppression of hand gestures upon detection of peripheral events on a peripheral device

Various implementations disclosed herein include devices, systems, and methods that interpret user activity as user interactions with user interface (UI) elements positioned within a three-dimensional (3D) space such as an extended reality (XR) environment. Some implementations enable user interactions with virtual elements displayed in 3D environments that utilize alternative input modalities, e.g., XR environments that interpret user activity as either direct interactions or indirect interactions with virtual elements.

Input recognition based on distinguishing direct and indirect user interactions

Various implementations disclosed interpret direct touch-based gestures, such as drag and scroll gestures, made by a user virtually touching one position of a user interface and moving their hand to another position of the user interface. For example, such gestures may be made relative to a user interface presented in an extended reality (XR) environment. In some implementations, a user movement is interpreted using one or more techniques that avoid unexpected gain or loss of user-interface-associated motion. Some implementations determine which segments of a movement to associate with user interface content based on characteristics of the movement. Some implementations determine that a break occurs when a user movement leaves a break volume that is adjusted dynamically.

Interpreting user movement as direct touch user interface interactions

Enabling gesture input includes obtaining hand tracking data based on one or more camera frames, detecting a contact event between a first finger and a second finger based on the hand tracking data, and determining a first contact location on the first finger and a second contact location on the second finger. In accordance with a determination that the first contact location and the second contact location are within a first predefined gesture zone for a first gesture, an input action is enabled corresponding to the first gesture.

Pinch recognition using finger zones

Growing community of inventors

San Jose, CA, United States of America

Daniel J Brewer