The Penn State Research Foundation

Adobe, Inc.

Jianbo Ye

James Z Wang

Jia Li

Zhe Lin

Xin Lu

Reginald B Adams

Yu Luo

An emotion analysis and recognition system including an automated recognition of bodily expression of emotion (ARBEE) system is described. The system may include program instructions executable by a processor to: receive a plurality of body movement models, each body movement model generated based on a crowdsourced body language dataset, calculate at least one evaluation metric for each body movement model, select a highest ranked body movement model based on the at least one metric calculated for each body movement model, combine the highest ranked body movement model with at least one other body movement model of the plurality of body movement models, calculate at least one evaluation metric for each combination of body movement models, and determine a highest ranked combination of body movement models to predict a bodily expression of emotion.

Systems and methods for automated recognition of bodily expression of emotion

The architectural complexity of a neural network is reduced by selectively pruning channels. A cost metric for a convolution layer is determined. The cost metric indicates a resource cost per channel for the channels of the layer. Training the neural network includes, for channels of the layer, updating a channel-scaling coefficient based on the cost metric. The channel-scaling coefficient linearly scales the output of the channel. A constant channel is identified based on the channel-scaling coefficients. The neural network is updated by pruning the constant channel. Model weights are updated via a stochastic gradient descent of a training loss function evaluated on training data. The channel-scaling coefficients are updated via an iterative-thresholding algorithm that penalizes a batch normalization loss function based on the cost metric for the layer and a norm of the channel-scaling coefficients. When the layer is batch normalized, the channel-scaling coefficients are batch normalization scaling coefficients.

Reducing architectural complexity of convolutional neural networks via channel pruning

A method of improving quality of crowdsourced affective data based on agreement relationship between a plurality of annotators include receiving, by a processor, a collection of stimuli previously given affective labels by the plurality of annotators, executing, by a processor, an algorithm operative to perform the steps including constructing an agreement multigraph as a probabilistic model including a pair-wise status of agreement between the affective labels given by different ones of the plurality of annotators, learning the probabilistic model computationally using the crowdsourced affective data, identifying a reliability of each of the plurality of annotators based on the learned model, and adjusting the crowdsourced affective data by calculating the affective labels of each stimuli based on the identified reliability of each of the plurality of annotators, thereby improving the quality of the crowdsourced affective data.

Probabilistic multigraph modeling for improving the quality of crowdsourced affective data

Computationally efficient accelerated D2-clustering algorithms are disclosed for clustering discrete distributions under the Wasserstein distance with improved scalability. Three first-order methods include subgradient descent method with re-parametrization, alternating direction method of multipliers (ADMM), and a modified version of Bregman ADMM. The effects of the hyper-parameters on robustness, convergence, and speed of optimization are thoroughly examined. A parallel algorithm for the modified Bregman ADMM method is tested in a multi-core environment with adequate scaling efficiency subject to hundreds of CPUs, demonstrating the effectiveness of AD2-clustering.

Growing community of inventors

State College, PA, United States of America

Jianbo Ye