Xerox Corporation

Conduent Business Services, LLC

Adrien Gaidon

Eleonora Vig

Florent C Perronnin

Albert Gordo Soldevila

Diane Larlus-Larrondo

José Antonio Rodriguez Serrano

Gloria Zen

Antonio M Lopez

Qiao Wang

César Roberto De Souza

A system for applying video data to a neural network (NN) for online multi-class multi-object tracking includes a computer programed to perform an image classification method including the operations of receiving a video sequence; detecting candidate objects in each of a previous and a current video frame; transforming the previous and current video frames into a temporal difference input image; applying the temporal difference input image to a pre-trained neural network (NN) (or deep convolutional network) comprising an ordered sequence of layers; and based on a classification value received by the neural network, associating a pair of detected candidate objects in the previous and current frames as belonging to one of matching objects and different objects.

Deep data association for online multi-class multi-object tracking

A system and method are suited for assessing video performance analysis. A computer graphics engine clones real-world data in a virtual world by decomposing the real-world data into visual components and objects in one or more object categories and populates the virtual world with virtual visual components and virtual objects. A scripting component controls the virtual visual components and the virtual objects in the virtual world based on the set of real-world data. A synthetic clone of the video sequence is generated based on the script controlling the virtual visual components and the virtual objects. The real-world data is compared with the synthetic clone of the video sequence and a transferability of conclusions from the virtual world to the real-world is assessed based on this comparison.

Generating a virtual world to assess real-world video analysis performance

Methods and systems for online domain adaptation for multi-object tracking. Video of an area of interest can be captured with an image-capturing unit. The video (e.g., video images) can be analyzed with a pre-trained object detector utilizing online domain adaptation including convex multi-task learning and an associated self-tuning stochastic optimization procedure to jointly adapt online all trackers associated with the pre-trained object detector and a pre-trained category-level model from the trackers in order to efficiently track a plurality of objects in the video captured by the image-capturing unit.

Online domain adaptation for multi-object tracking

A computer-implemented video classification method and system are disclosed. The method includes receiving an input video including a sequence of frames. At least one transformation of the input video is generated, each transformation including a sequence of frames. For the input video and each transformation, local descriptors are extracted from the respective sequence of frames. The local descriptors of the input video and each transformation are aggregated to form an aggregated feature vector with a first set of processing layers learned using unsupervised learning. An output classification value is generated for the input video, based on the aggregated feature vector with a second set of processing layers learned using supervised learning.

System and method for video classification using a hybrid unsupervised and supervised multi-layer architecture

A method for extracting a representation from an image includes inputting an image to a pre-trained neural network. The gradient of a loss function is computed with respect to parameters of the neural network, for the image. A gradient representation is extracted for the image based on the computed gradients, which can be used, for example, for classification or retrieval.

Extracting gradient features from neural networks

An object detection method includes for each of a set of patches of an image, encoding features of the patch with a non-linear mapping function, and computing per-patch statistics based on the encoded features for approximating a window-level non-linear operation by a patch-level operation. Then, windows are extracted from the image, each window comprising a sub-set of the set of patches. Each of the windows is scored based on the computed patch statistics of the respective sub-set of patches. Objects, if any, can then be detected in the image, based on the window scores. The method and system allow the non-linear operations to be performed only at the patch level, reducing the computation time of the method, since there are generally many more windows than patches, while not impacting performance unduly, as compared to a system which performs non-linear operations at the window level.

Efficient object detection with patch-level window processing

A tracking system and method are suited to tracking multiple of objects of different categories in a video sequence. A sequence of video frames is received and a set of windows is extracted from each frame in turn, based on a computed probability that the respective window contains an object, without reference to any specific category of object. For each of these windows, a feature representation is extracted. A trained detector for a selected category detects windows that constitute targets in that category, based on the respective feature representations. More than one detector can be used when there is more than one category of objects to be tracked. A target-specific appearance model is generated for each of the targets (e.g., learned or updated, if the target is present in a prior frame). The detected targets are tracked over one or more subsequent frames based on the target-specific appearance models of the targets.

Multi-object tracking with generic object proposals

A system and method enable generating a specific object detector for a category of interest. The method includes identifying seed objects in frames of a video sequence with a pre-trained generic detector for the category. An appearance model is iteratively learned for each of the seed objects using other frames in which the seed object is identified. The appearance models are learned jointly to optimize a loss function which accounts for the loss of incorrectly labeling sub-images and a regularization term which measures a distance between the appearance models. The loss of incorrectly labeling sub-images is determined using a motion model which predicts the location of the seed object in the subsequent frames so that sub-images outside the location that the current appearance model contribute to the loss. The specific object detector is then generated by aggregating the optimized appearance models.

Growing community of inventors

Grenoble, France

Adrien Gaidon