System and method for video analytics processing on dynamical content

Abstract

A multi-frame video analytics (MFVA) system is disclosed, in which contiguous video frame-sequences are processed in reconstruction of 3D dynamical relationships amongst tracked objects. A tensor-based spatiotemporal volume (STV) formalism mapped to buffered successive video-frames is employed. An optical flow field calculation is then applied to this STV for recovery of z-axis information. Standard dynamics in terms of position, velocity, and acceleration are then reconstructed via numerical approximation of time-derivatives on the set of spatiotemporal trajectories (STT) generated by tracked objects that together constitute a bounding STV. In this manner, STVs are rendered causal, with implication application of a suitable knowledge (KBASE) representation also generates an explanatory framework consisting of a set of primitive assertions on tracked objects, now informed by dynamical state, phase flow, and interaction events. This framework is then employed as basis for generation of an enhanced set of video analytics.