Microsoft Technology Licensing, LLC

Nahil Tawfik Sharkasi

Julia Schwarz

Casey Leon Meekhof

Sheng Kai Tang

Thomas Matthew Gable

Nicholas Ferianc Kamuda

Sophie Stellmach

Ramiro Torres

Joshua Kyle Neff

Jamie Bryant Kirschenbaum

Daniel Joseph McCulloch

Brian James Mount

Shawn Crispin Wright

Anthony James Ambrus

Kyungsuk David Lee

Alton Kwok

Jason Michael Ray

Daniel Lee Osborn

David C Haley, Jr

Albert Robles

Dave Hill

Kevin John Appel

Andrew R Campbell

Alexandru Octavian Balan

Chuan Qin

Neil Richard Kronlage

Ramiro S Torres

A computing system is provided. The computing system includes a head mounted display (HMD) device including a display, a processor configured to execute one or more programs, and associated memory. The processor is configured to display a virtual object at least partially within a field of view of a user on the display, identify a plurality of control points associated with the virtual object, and determine that one or more of the control points associated with the virtual object are further than a predetermined threshold distance from the user. The processor is configured to, based on the determination, invoke a far interaction mode for the virtual object and receive a trigger input from the user. In response to the trigger input in the far interaction mode, the processor is configured to invoke a near interaction mode and display a virtual interaction object within the predetermined threshold distance from the user.

Near interaction mode for far virtual object

A head-mounted display comprises a display device and an outward-facing depth camera. A storage machine comprises instructions executable by a logic machine to present one or more virtual objects on the display device, to receive information from the depth camera about an environment, and to determine a position of the head-mounted display within the environment. Based on the position of the head-mounted display, a position of a joint of a user's arm is inferred. Based on the information received from the depth camera, a position of a user's hand is determined. A ray is cast from a portion of the user's hand based on the position of the joint of the user's arm and the position of the user's hand. Responsive to the ray intersecting with one or more control points of a virtual object, the user is provided with an indication that the virtual object is being targeted.

Holographic palm raycasting for targeting virtual objects

Systems and methods are provided for selectively enabling or disabling control rays in mixed-reality environments. In some instances, a mixed-reality display device presents a mixed-reality environment to a user which includes one or more holograms. The display device then detects a user gesture input associated with a user control (which may include a part of the user's body) during presentation of the mixed-reality environment. In response to detecting the user gesture, the display device selectively generates and displays a corresponding control ray as a hologram rendered by the display device extending away from the user control within the mixed-reality environment. Gestures may also be detected for selectively disabling control rays so that they are no longer rendered.

Discrete and continuous gestures for enabling hand rays

A method for augmenting a two-stage hand gesture input comprises receiving hand tracking data for a hand of a user. A gesture recognition machine recognizes that the user has performed a first-stage gesture based on one or more parameters derived from the received hand tracking data satisfying first-stage gesture criteria. An affordance cueing a second-stage gesture is provided to the user responsive to recognizing the first-stage gesture. The gesture recognition machine recognizes that the user has performed the second-stage gesture based on one or more parameters derived from the received hand tracking data satisfying second-stage gesture criteria. A graphical user interface element is displayed responsive to recognizing the second-stage gesture.

Augmented two-stage hand gesture input

A method for improving user interaction with a virtual environment includes presenting the virtual environment to a user on a display, measuring a gaze location of a user's gaze relative to the virtual environment, casting an input ray from an input device, measuring an input ray location at a distal point of the input ray, and snapping a presented ray location to the gaze location when the input ray location is within a snap threshold distance of the input ray location.

Systems and methods for using eye gaze to bend and snap targeting rays for remote interaction

A system and method are disclosed relating to a pipeline for generating a computer model of a target user, including a hand model of the user's hands, captured by an image sensor in a NUI system. The computer model represents a best estimate of the position of a user's hand or hands and whether the hand or hand is in an open or closed state. The generated hand model may be used by a gaming or other application to determine such things as user gestures and control actions.

System for recognizing an open or closed hand

A system and method are disclosed relating to a pipeline for generating a computer model of a target user, including a hand model of the user's hands and fingers, captured by an image sensor in a NUI system. The computer model represents a best estimate of the position and orientation of a user's hand or hands. The generated hand model may be used by a gaming or other application to determine such things as user gestures and control actions.

System for finger recognition and tracking

Growing community of inventors

Woodinville, WA, United States of America

Nahil Tawfik Sharkasi