Systems and methods for determining hand poses in artificial reality environments

Abstract

A method includes a computing system receiving an image of a real-world environment, the image including at least a portion of a hand of a user of an artificial reality device, the hand comprising a palm and a plurality of fingers. The computing system determines a hand pose of the hand using the image, and defines, based on the hand pose, a three-dimensional surface positioned in the palm of the hand. The computing system determines, based on the hand pose, distances between predetermined portions of the plurality of fingers and the three-dimensional surface. The computing system assigns, based on the distances, a pose value for each of the plurality of fingers of the hand and determines, based on the pose values for the plurality of fingers, a grab state of the hand.