International Business Machines Corporation

Ankur Datta

Sharathchandra UmapathiRao Pankanti

Rogerio Schmidt Feris

Yun Zhai

Lisa Maire Brown

Xiaoyu Wang

Daniel Andre Vaquero

Behjat Siddiquie

Balamanohar Paluri

Charles A Otto

Sharathchandra Umapathirao Pankanti

Aspects determining anomalous events, wherein processors determine a trajectory of tracked movement of an object through an image field of a camera partitioned into a matrix grid of different local units. The aspects generate anomaly confidence decision values for image features extracted from video data of the tracked movement of the object as a function of fitting extracted image features to normal patterns of local motion pattern models defined by dominant distributions of extracted image features. The aspects further extract trajectory features from the video data relative to the trajectory of the tracked movement of the object, and generate global anomaly confidence decision values for the object trajectory as a function of fitting the extracted trajectory features to a normal learned motion trajectory model. The aspects determine anomalous events as a function of the generated global anomaly confidence decision value and the anomaly confidence decision values.

Anomalous event retriever

An approach for re-identifying an object in a test image is presented. Similarity measures between the test image and training images captured by a first camera are determined. The similarity measures are based on Bhattacharyya distances between feature representations of an estimated background region of the test image and feature representations of background regions of the training images. A transformed test image based on the Bhattacharyya distances has a brightness that is different from the test image's brightness and matches a brightness of training images captured by a second camera. An appearance of the transformed test image resembles an appearance of a capture of the test image by the second camera. Another image included in test images captured by the second camera is identified as being closest in appearance to the transformed test image and another object in the identified other image is a re-identification of the object.

Re-identifying an object in a test image

An approach for re-identifying an object in a test image is presented. Similarity measures between the test image and training images captured by a first camera are determined. The similarity measures are based on Bhattacharyya distances between feature representations of an estimated background region of the test image and feature representations of background regions of the training images. A transformed test image based on the Bhattacharyya distances has a brightness that is different from the test image's brightness, and matches a brightness of training images captured by a second camera. An appearance of the transformed test image resembles an appearance of a capture of the test image by the second camera. Another image included in test images captured by the second camera is identified as being closest in appearance to the transformed test image and another object in the identified other image is a re-identification of the object.

Foreground objects of interest are distinguished from a background model by dividing a region of interest of a video data image into a grid array of individual cells. Each of the cells are labeled as foreground if accumulated edge energy within the cell meets an edge energy threshold, or if color intensities for different colors within each cell differ by a color intensity differential threshold, or as a function of combinations of said determinations.

Multi-cue object detection and analysis

Image-matching tracks the movements of the objects from initial camera scenes to ending camera scenes in non-overlapping cameras. Paths are defined through scenes for pairings of initial and ending cameras by different respective scene entry and exit points. For each of said camera pairings a combination path having a highest total number of tracked movements relative to all other combinations of one path through the initial and ending camera scene is chosen, and the scene exit point of the selected path through the initial camera and the scene entry point of the selected path into the ending camera define a path connection of the initial camera scene to the ending camera scene.

Topology determination for non-overlapping camera network

Local models learned from anomaly detection are used to rank detected anomalies. The local model patterns are defined from image feature values extracted from an image field of video image data with respect to different predefined spatial and temporal local units, wherein anomaly results are determined by fitting extracted image features to the local model patterns. Image features values extracted from the image field local units associated with anomaly results are normalized, and image feature values extracted from the image field local units are clustered. Weights for anomaly results are learned as a function of the relations of the normalized extracted image feature values to the clustered image feature values. The normalized values are multiplied by the learned weights to generate ranking values to rank the anomalies.

Efficient retrieval of anomalous events with priority learning

A camera at a fixed vertical height positioned above a reference plane, with an axis of a camera lens at an acute angle with respect to a perpendicular of the reference plane. One or more processors receive camera images of a multiplicity of people of unknown height and vertical axis of the images are transformed into pixel counts. The known heights of people from a known statistical distribution of heights of people are received by one or more processors and transformed to a normalized measurement of pixel counts, based in part on a focal length of the camera lens, the angle of the camera, and an objective function summing differences between pixel counts of the known heights of people and the unknown heights of people. The fixed vertical height of the camera is determined by adjusting the estimated camera height to minimize the objective function.

Determining camera height using distributions of object heights and object image heights

Objects within two-dimensional video data are modeled by three-dimensional models as a function of object type and motion through manually calibrating a two-dimensional image to the three spatial dimensions of a three-dimensional modeling cube. Calibrated three-dimensional locations of an object in motion in the two-dimensional image field of view of a video data input are determined and used to determine a heading direction of the object as a function of the camera calibration and determined movement between the determined three-dimensional locations. The two-dimensional object image is replaced in the video data input with an object-type three-dimensional polygonal model having a projected bounding box that best matches a bounding box of an image blob, the model oriented in the determined heading direction. The bounding box of the replacing model is then scaled to fit the object image blob bounding box, and rendered with extracted image features.

Estimation of object properties in 3D world

Field of view overlap among multiple cameras are automatically determined as a function of the temporal overlap of object tracks determined within their fields-of-view. Object tracks with the highest similarity value are assigned into pairs, and portions of the assigned object track pairs having a temporally overlapping period of time are determined. Scene entry points are determined from object locations on the tracks at a beginning of the temporally overlapping period of time, and scene exit points from object locations at an ending of the temporally overlapping period of time. Boundary lines for the overlapping fields-of-view portions within the corresponding camera fields-of-view are defined as a function of the determined entry and exit points in their respective fields-of-view.

Field of view determiner

An approach for re-identifying an object in a first test image is presented. Brightness transfer functions (BTFs) between respective pairs of training images are determined. Respective similarity measures are determined between the first test image and each of the training images captured by the first camera (first training images). A weighted brightness transfer function (WBTF) is determined by combining the BTFs weighted by weights of the first training images. The weights are based on the similarity measures. The first test image is transformed by the WBTF to better match one of the training images captured by the second camera. Another test image, captured by the second camera, is identified because it is closer in appearance to the transformed test image than other test images captured by the second camera. An object in the identified test image is a re-identification of the object in the first test image.

Appearance modeling for object re-identification using weighted brightness transfer functions

Images are retrieved and ranked according to relevance to attributes of a multi-attribute query through training image attribute detectors for different attributes annotated in a training dataset. Pair-wise correlations are learned between pairs of the annotated attributes from the training dataset of images. Image datasets may are searched via the trained attribute detectors for images comprising attributes in a multi-attribute query. The retrieved images are ranked as a function of comprising attributes that are not within the query subset plurality of attributes but are paired to one of the query subset plurality of attributes by the pair-wise correlations, wherein the ranking is an order of likelihood that the different ones of the attributes will appear in an image with the paired one of the query subset plurality of attributes.

Image ranking based on attribute correlation

In response to a query of discernable facial attributes, the locations of distinct and different facial regions are estimated from face image data, each relevant to different attributes. Different features are extracted from the estimated facial regions from database facial images, which are ranked in base layer rankings as a function of relevance of extracted features to attributes relevant to the estimated regions, and in second-layer rankings as a function of combinations of the base layer rankings and relevance of the extracted features to common ones of the attributes relevant to the estimated regions. The images are ranked in relevance to the query as a function of the second-layer rankings.

Hierarchical ranking of facial attributes

A camera at a fixed vertical height positioned above a reference plane, with an axis of a camera lens at an acute angle with respect to a perpendicular of the reference plane. One or more processors receive images of different people. The vertical measurement values of the images of different people are determined. The one or more processors determine a first statistical measure associated with a statistical distribution of the vertical measurement values. The known heights of people from a known statistical distribution of heights of people are transformed to normalized measurements, based in part on a focal length of the camera lens, the angle of the camera, and a division operator in an objective function of differences between the normalized measurements and the vertical measurement values. The fixed vertical height of the camera is determined, based at least on minimizing the objective function.

Growing community of inventors

White Plains, NY, United States of America

Ankur Datta