Machine learning model-based sign language corrective feedback

Abstract

In various examples, machine learning model-based sign language corrective feedback is provided. Machine learning model-based recognition of sign language symbols (e.g., body poses and/or movements) may be used to generate real-time kinematic feedback to a sign language speaker that guides them to adjust their hand pose to correctly align with established sign language standard symbols. A sign language feedback framework may process video data representing conversational sign language symbol sequences to extract kinematic keypoints to search a sign language dictionary representing an established vocabulary of sign language symbols. Based on selecting an intended sign language symbol from the dictionary and computing deviations between the kinematic keypoint pattern of the intended sign language symbol from the dictionary and the extracted kinematic keypoint symbol, the sign language feedback framework may provide real-time kinematic feedback to the signer describing how to adjust their signing.