Canon Kabushiki Kaisha

Nagita Mehrseresht

Alan Valev Tonisson

Andrew James Dorrell

Barry James Drake

A method of classifying an interaction captured in a sequence of video. A plurality of people in the video sequence is identified. An action of a first one of the people at a first time is determined. An action of a second one of the people at a second time is determined, the action of the second person being after the action of the first person. A role for the second person at the second time is determined, the role being independent of the determined actions of the first and second person. An interaction between the first person and the second person is classified based on the determined role of the second person and the determined actions of the first and second person.

Interaction classification using the role of people interacting over time

A method and associated imaging system for classifying at least one concept type in a video segment is disclosed. The method associates an object concept type in the video segment with a spatio-temporal segment of the video segment. The method then associates a plurality of action concept types with the spatio-temporal segment, where each action concept type of the plurality of action concept types is associated with a subset of the spatio-temporal segment associated with the object concept type. The method then classifies the action concept types and the object concept types associated with the video segment using a conditional Markov random field (CRF) model where the CRF model is structured with the plurality of action concept types being independent and indirectly linked via a global concept type assigned to the video segment, and the object concept type is linked to the global concept type.

Imaging system and method for classifying a concept type in video

A system and method of detecting an interaction between a plurality of objects. The method comprises receiving tracking information for the plurality of objects in a scene; generating a plurality of frames, each of the plurality of frames comprising an activation for each of the plurality of objects and representing a relative spatial relationship between the plurality objects in the scene determined from the received tracking information, the frames encoding properties of the objects using properties of the corresponding activations; determining, using a trained neural network, features associated with the plurality of objects from the generated plurality of frames using the activations and the relative spatial relationship between the objects, the features representing changes in the relative spatial relationship between the objects over time relating to the interaction; and detecting time localization of the interaction in the plurality of frames using the determined features.

System and method for detecting interaction

A method of determining a composite action from a video clip, using a conditional random field (CRF), the method includes determining a plurality of features from the video clip, each of the features having a corresponding temporal segment from the video clip. The method may continue by determining, for each of the temporal segments corresponding to one of the features, an initial estimate of an action unit label from a corresponding unary potential function, the corresponding unary potential function having as ordered input the plurality of features from a current temporal segment and at least one other of the temporal segments. The method may further include determining the composite action by jointly optimizing the initial estimate of the action unit labels.

Tree structured CRF with unary potential function using action unit features of other segments as context feature

A method of jointly classifying a plurality of objects in an image using a feature type selected from a plurality of feature types determines classification information for each of the plurality of objects in the image by applying a predetermined joint classifier to at least one feature of a first type. The feature is generated from the image using a first feature extractor, the classification information being based on a probability of each of a plurality of possible classifications. The method estimates, for each of the feature types, an improvement in an accuracy of classification for each of the plurality of objects. The method selects features of a further type, from the plurality of feature types, according to the estimated improvement in the accuracy of the classification of each of the objects, and classifies the plurality of objects in the image using the selected features of the further type.

Dynamic feature selection for joint probabilistic recognition

A method of segmenting a video sequence. A segment score is determined for each of a plurality of fixed length segments of the video sequence. Each of the segment scores provide a score for a plurality of actions associated with a corresponding fixed length segment. A current segment is selected from the segments of the video sequence. The segment score is selected for a further one of the segments, the further segment being disjoint with the current segment and being used to provide information about actions that were classified outside the current segment. A further segment score is determined for the current segment according to the selected segment score. The video is segmented based on the determined further segment score.

Temporal segmentation of actions using context features

A method of determining interpolation coefficients () of a symmetric interpolation kernel () is disclosed. The method comprises determining a first interpolation coefficient () from the symmetric interpolation kernel () and storing the first interpolation coefficient in a memory (). The method then determines the value of an intermediate function () from symmetrically opposed segments () of the kernel, and determines a subsequent interpolation coefficient dependent upon the first interpolation coefficient and the value of the intermediate function.

Efficient kernel calculation for interpolation

A method () of determining an image value at a sample position of an output image, is disclosed. The method () comprises the steps of determining orientation of an isophote (e.g.,) passing through the output sample position and determining a period of intensity variation along the isophote (). The method () determines the image value at the sample position of the output image based on the period of intensity variation and outputs the determined image value at the sample position of the output image.

Growing community of inventors

West Pennant Hills, Australia

Nagita Mehrseresht