Adobe, Inc.

The Penn State Research Foundation

Yu Luo

Jen-Chan Jeff Chien

Xin Lu

James Z Wang

Jianbo Ye

Reginald B Adams

An emotion analysis and recognition system including an automated recognition of bodily expression of emotion (ARBEE) system is described. The system may include program instructions executable by a processor to: receive a plurality of body movement models, each body movement model generated based on a crowdsourced body language dataset, calculate at least one evaluation metric for each body movement model, select a highest ranked body movement model based on the at least one metric calculated for each body movement model, combine the highest ranked body movement model with at least one other body movement model of the plurality of body movement models, calculate at least one evaluation metric for each combination of body movement models, and determine a highest ranked combination of body movement models to predict a bodily expression of emotion.

Systems and methods for automated recognition of bodily expression of emotion

Techniques are disclosed for a facial expression classification. In an embodiment, a multi-class classifier is trained using labelled training images, each training image including a facial expression. The trained classifier is then used to predict expressions for unlabelled video frames, whereby each frame is effectively labelled with a predicted expression. In addition, each predicted expression can be associated with a confidence score. Anchor frames can then be identified in the labelled video frames, based on the confidence scores of those frames (anchor frames are frames having a confidence score above an established threshold). Then, for each labelled video frame between two anchor frames, the predicted expression is refined or otherwise updated using interpolation, thereby providing a set of video frames having calibrated expression labels. These calibrated labelled video frames can then be used to further train the previously trained facial expression classifier, thereby providing a supplementally trained facial expression classifier.

Facial expression recognition utilizing unsupervised learning

Growing community of inventors

State College, PA, United States of America

Yu Luo