The Hong Kong University of Science and Technology

Huawei Technologies Co., Limited

Blue Planet Training, Inc.

Huamin Qu

Yong Wang

Mingxuan Yuan

Bing Ni

Aoyu Wu

Huan Wei

Haotian Li

Quan Li

Conglei Shi

Qing Chen

Linping Yuan

Haipeng Zeng

Siwei Fu

Yuanzhe Chen

Xingbo Wang

Min Xu

The invention relates to a computer implemented method and system for analyzing a video. The method comprises the steps of receiving, via a receiving module, a video data comprising a series of images showing a subject; extracting, via an extracting module, a transcript derived from an audio data associated with the video data; aligning, via an aligning module, the series of images of the video data with the transcript derived from the audio data associated with the video data based on timestamps derived from the video data; analyzing, via an analyzing module, gestures of the subject from the series of images, comprising the steps of: identifying a plurality of reference points from each of the series of images showing the subject; segmenting the series of images in accordance with one or more selected texts comprising the transcript; identifying a defined gesture type for each of the segmented images based on the identified plurality of reference points; and processing, via a processing module, the defined gesture types in correlation with respective one or more selected texts comprising the transcript.

System and a method for analyzing a video

A method for predicting student performance in interactive question pools, comprising: a data processing and feature extraction module for: extracting from historical score data records statistical student features reflecting the students' past performances on the questions; and statistical question features indicating a question popularity of each question and the average students' scores on each question; and extracting from the pointing device movement data records a plurality of interactive edge features representing the characteristics of the students' problem-solving behaviors; a network construction module for: performing an Edge2Node transformation to transform the interactive edge features into one or more interaction nodes; and constructing a heterogeneous student-interaction-question (SIQ) network; and a prediction module for processing the SIQ network using a residual relational graph neural network to predict a student's score on unattempt questions.

Peer-inspired student performance prediction in interactive online question pools with graph neural network

A system for proctoring online exams includes a client-side computing system and a visual analytics system. The client-side computing system includes a camera configured to obtain video data corresponding to a user while taking an exam and one or more input devices configured to obtain interaction data, wherein the interaction data includes mouse movements of the user while taking the exam. The visual analytics system is configured to obtain the video data and the interaction data from the client-side computing system, analyze the exam data to detect abnormal behavior by the user based at least in part on mouse movement data, and generate one or more visualizations of the analyzed exam data to be used in determining whether or not the user has cheated during the exam.

Visual analytics tool for proctoring online exams

A computer implemented method and system for processing an audio signal. The method includes the steps of extracting prosodic features from the audio signal, aligning the extracted prosodic features with a script derived from or associated with the audio signal, and segmenting the script with the aligned extracted prosodic features into structural blocks of a first type. The method may further include determining a distance measure between a structural block of a first type derived from the script with another structural block of the first type using, for example, the Damerau-Levenshtein distance.

System and a method for speech analysis

A computer implemented method and system processing a video signal. The method comprises comprising the steps of: detecting a human face displayed in the video signal and extracting physiological, biological, or behavior state information from the displayed face at a first level of granularity of the video signal; processing any two or more of: (i) a script derived from or associated with the video signal to extract language tone information from said script at a first level of granularity of the script; (ii) an audio signal derived from or associated with the video signal to derive behavior state information from said audio signal at a first level of granularity of the audio signal; (iii) a video image derived from the video signal to detect one or more human gestures of the person whose face is displayed in the video signal; and merging said physiological, biological, or behavior state information extracted from the displayed face in the video signal with any two or more of: (i) the language tone information extracted from the script; (ii) the behavior state information derived from the audio signal; and (iii) and the one or more human gestures derived from the video image, wherein the merging step is based on behavior state categories and/or levels of granularity.

System and method for visual analysis of emotional coherence in videos

The present teaching relates to analyzing user activities related to a video. The video is provided to a plurality of users. The plurality of users is monitored to detect one or more types of user activities performed in time with respect to different portions of the video. One or more visual representations of the monitored one or more types of user activities are generated. The one or more visual representations capture a level of attention paid by the plurality of users to the different portions of the video at any time instance. Interests of at least some of the plurality of users are determined with respect to the different portions of the video based on the one or more visual representations.

Method and system for analyzing user activities related to a video

The present disclosure provides a data visualization method and apparatus. The method includes: displaying a distribution curve diagram by using a display device, where the distribution curve diagram is used to represent time-based distribution of multiple pieces of spatial-temporal data, and the distribution curve diagram includes a timeline, a curve changing with the timeline, and an irregular region enclosed by the curve; and displaying a tree map in the irregular region by using the display device, where the tree map divides the irregular region into m sub-regions that are in a one-to-one correspondence with m regions, and an area of each of the m sub-regions is used to indicate an amount of spatial-temporal data distributed in a region corresponding to each sub-region. The present disclosure implements visualization of spatial-temporal data by using the tree map.

Data visualization method and apparatus

A data visualization method includes acquiring a spatiotemporal behavior data set generated by network users, determining, according to a time point at which or a time period in which each piece of spatiotemporal behavior data is generated, an amount of spatiotemporal behavior data that is in the spatiotemporal behavior data set and generated at a time point or in a time period falling within each time period in N time periods, and presenting a column bar corresponding to an itime period in the N time periods, where the column bar is in a radial direction of a first circle representing a full time domain and intersects the first circle and the length of a column bar represents a quantity of behaviors in a time period corresponding to the column bar.

Growing community of inventors

Hong Kong, China

Huamin Qu