Princeton Video Image, Inc.

Sri International

Sarnoff Corporation

Subhodev Das

Yi Tan

Roy Jonathan Rosser

Harpreet Singh Sawhney

Rakesh Kumar

Supun Samarasekera

Brian Astle

Ajay Divakaran

Manoj Aggarwal

Feng Han

Ming-Yee Chiu

Andrew Coppock

Peter Von Kaenel

Julia Kruk

Bo Dong

In general, the disclosure describes techniques for joint spatiotemporal Artificial Intelligence (AI) models that can encompass multiple space and time resolutions through self-supervised learning. In an example, a method includes for each of a plurality of multimodal data, generating, by a computing system, using a first machine learning model, a respective modality feature vector representative of content of the multimodal data, wherein each of the generated modality feature vectors has a different modality; processing, by the computing system, each of generated modality feature vectors with a second machine learning model comprising an encoder model to generate event data comprising a plurality of events and/or activities of interest; and analyzing, by the computing system, the event data to generate anomaly data indicative of detected anomalies in the multimodal data.

Spatial-temporal anomaly and event detection using night vision sensors

A system and method for detecting a target in imagery is disclosed. At least one image region exhibiting changes in at least intensity is detected from among at least a pair of aligned images. A distribution of changes in at least intensity inside the at least one image region is determined using an unsupervised learning method. The distribution of changes in at least intensity is used to identify pixels experiencing changes of interest. At least one target from the identified pixels is identified using a supervised learning method. The distribution of changes in at least intensity is a joint hue and intensity histogram when the pair of images pertain to color imagery. The distribution of changes in at least intensity is an intensity histogram when the pair of images pertain to grey-level imagery.

Method and apparatus for detecting targets through temporal scene changes

Method for tracking an object recorded within a selected frame of a sequence of frames of video data, using a plurality of layers, where at least one object layer of the plurality of layers represents the object includes initializing layer ownership probabilities for pixels of the selected frame using a non-parametric motion model, estimating a set of motion parameters of the plurality of layers for the selected frame using a parametric maximization algorithm and tracking the object. The non-parametric motion model is optical flow and includes warping the mixing probabilities, the appearances of the plurality of layers, and the observed pixel data from the pixels of the preceding frame to the pixels of the selected frame to initialize the layer ownership probabilities for the pixels of the selected frame.

Flexible layer tracking with weak online appearance model

A system and method for realtime occlusion processing for seamlessly and realistically blending an inserted image such as an advertisement into a region of a live broadcast image without obscuring the action of the live image. The average color and intensity of a synthetic reference image containing at least some of the region to be replaced is compared to the average color and intensity of the current live broadcast image to determine the difference between the two images. The resulting difference image obtained from processing the current image and synthetic, reference image determines areas of the intended insertion region within the current image which are obscured by live action. The processor then generates an occlusion mask based on the difference image and only those pixels that are unoccluded within the intended insertion region are allowed to be inserted into the live broadcast.

System and method of real time insertions into video using adaptive

A method for tracking motion from field to field in a sequence of related video broadcast images. The method uses template correlation to follow a set of predetermined landmarks within a scene in order to provide position information of objects in the current image. The current image object position information is compared to position information of the same objects within a reference array data table. The comparison is accomplished through the use of warp equations that map points in the current image to points in the reference array. Motion is tracked according to a velocity prediction scheme utilizing a weighted formula that emphasizes the weight of landmarks that are closer to their predicted position.

Method of tracking scene motion for live video insertion systems

A system for inserting images into live video fields includes a method for rapidly and efficiently identifying landmarks and objects. Initially a first template, having a first pattern similar to one of the distinctive features of the object, is passed over the video field and compared to it in order to preliminarily identify at least one possible distinctive feature as a candidate. A second template is then created by taking one of the major elements of the distinctive feature candidate and extending that element all the way across the second template and then comparing it to the distinctive feature candidate. This eliminates one or more possible falsely identified features. A third template is then created having a pattern formed from another major element of said distinctive feature and extending it all the way across the third template. The third template is then likewise passed over the distinctive feature candidate and compared therewith in order to eliminate still further falsely identified features. The method is continued until all possible false alarm candidates have been eliminated. The process is then repeated in order to preliminarily identify two or three landmarks of the target object. The locations of those objects are then compared to a geometric model to further verify if the object has been correctly identified. The methodology can be tested against a video taped program to determine if it accurately identifies objects.

Pattern recognition system employing unlike templates to detect objects

Growing community of inventors

Princeton, NJ, United States of America

Subhodev Das