University System of Maryland

Adobe, Inc.

Trisha Mittal

Dinesh Manocha

Uttaran Bhattacharya

Rohan Chandra

Aniket Bera

Viswanathan Swaminathan

Ritwik Sinha

Saayan Mitra

John Philip Collomosse

Somdeb Sarkhel

David Arbour

Pooja Guhan

Techniques for video manipulation detection are described to detect one or more manipulations present in digital content such as a digital video. A detection system, for instance, receives a frame of a digital video that depicts at least one entity. Coordinates of the frame that correspond to a gaze location of the entity are determined, and the detection system determines whether the coordinates correspond to a portion of an object depicted in the frame to calculate a gaze confidence score. A manipulation score is generated that indicates whether the digital video has been manipulated based on the gaze confidence score. In some examples, the manipulation score is based on at least one additional confidence score.

Video manipulation detection

A video is classified as real or fake by extracting facial features, including facial modalities and facial emotions, and speech features, including speech modalities and speech emotions, from the video. The facial and speech modalities are passed through first and second neural networks, respectively, to generate facial and speech modality embeddings. The facial and speech emotions are passed through third and fourth neural networks, respectively, to generate facial and speech emotion embeddings. A first distance, d, between the facial modality embedding and the speech modality embedding is generated, together with a second distance, d, between the facial emotion embedding and the speech emotion embedding. The video is classified as fake if a sum of the first distance and the second distance exceeds a threshold distance. The networks may be trained using real and fake video pairs for multiple subjects.

System and method for detecting fabricated videos

Various disclosed embodiments are directed to using one or more algorithms or models to select a suitable or optimal variation, among multiple variations, of a given content item based on feedback. Such feedback guides the algorithm or model to arrive at suitable variation result such that the variation result is produced as the output for consumption by users. Further, various embodiments resolve tedious manual user input requirements and reduce computing resource consumption, among other things, as described in more detail below.

Media content enhancement based on user feedback of multiple variations

Systems, methods, apparatuses, and computer program products for recognizing human emotion in images or video. A method for recognizing perceived human emotion may include receiving a raw input. The raw input may be processed to generate input data corresponding to at least one context. Features from the raw input data may be extracted to obtain a plurality of feature vectors and inputs. The plurality of feature vectors and the inputs may be transmitted to a respective neural network. At least some of the plurality of feature vectors may be fused to obtain a feature encoding. Additional feature encodings may be computed from the plurality of feature vectors via the respective neural network. A multi-label emotion classification of a primary agent may be performed in the raw input based on the feature encoding and the additional feature encodings.

Human emotion recognition in images or video

Systems, methods, apparatuses, and computer program products for providing multimodal emotion recognition. The method may include receiving raw input from an input source. The method may also include extracting one or more feature vectors from the raw input. The method may further include determining an effectiveness of the one or more feature vectors. Further, the method may include performing, based on the determination, multiplicative fusion processing on the one or more feature vectors. The method may also include predicting, based on results of the multiplicative fusion processing, one or more emotions of the input source.

Growing community of inventors

College Park, MD, United States of America

Trisha Mittal