Eastman-kodak Company

At+t Corp.

University of Rochester

Sharp Laboratories, Inc.

A Murat Tekalp

Atul Puri

Rajiv Mehrotra

A Tanju Erdem

Gozde Bozdagi

Candemir Toklu

Petrus J L Van Beck

Petrus J Van Beek

Yucel Altunbasak

An object-oriented method for describing the content of a video sequence comprises the steps of (a) establishing a temporal object-based segment for an object of interest; (b) describing the temporal object-based segment by describing one or more semantic motions of the object within its temporal object-based segment; and (c) describing the temporal object-based segment by describing one or more semantic interactions of the object with one or more other objects within its temporal object-based segment. The semantic motions of the object may be further described in terms of the properties of elementary coherent motions within the semantic motion. Additionally, the semantic interactions of the object may be further described in terms of the properties of the elementary spatio-temporal relationships among the interacting objects.

Method and system for object-oriented motion-based video description

A method and apparatus for coding video data permits coding of video information with optional, enhanced functionalities. Video data is coded as base layer data and enhancement layer data. The base layer data includes convention motion compensated transform encoded texture and motion vector data. Optional enhancement layer data contains mesh node vector data. Mesh node vector data of the enhancement layer may be predicted based on motion vectors of the base layer. Thus, simple decoders may decode the base layer data and obtain a basic representation of the coded video data. However, more powerful decoders may decode both the base layer and enhanced layer to obtain decoded video permitting functionalities. An embodiment of the present invention provides a back channel that permits a decoder to affect how mesh node coding is performed in the encoder. The decoder may command the encoder to reduce or eliminate encoding of mesh node motion vectors. The back channel finds application in single layer systems and two layer systems.

Mesh node motion coding to enable object based functionalities within a motion compensated transform video coder

Mesh node coding to enable object based functionalities within a motion

A method for motion tracking and constructing a mosaic of video objects is disclosed. Also disclosed is a method for synthetic object transfiguration from a mosaic. A 2-D triangular mesh is employed to represent a video object, which permits to describe the motion of the object by the displacements of the node points of the mesh, and to describe any intensity variations by the contrast and brightness parameters estimated for each node point. Using the temporal history of the node point locations, the nodes of the 2-D mesh are continued to be tracked even when they become invisible because of self-occlusion or occlusion by another object. By adding new nodes or updating the 2-D triangular mesh, any uncovered parts of the video object are incrementally added to the mosaic. A finite number of views representing the constructed mosaic are selected and used for synthetic transfiguration of a replacement object in place of the original one.

Method for object tracking and mosaicing in an image sequence using a

In a temporal sequence of digitized image frames of video signals, the spatial and temporal image gradients and the pixel-to-pixel motion vectors (dense motion vectors) are obtained between two consecutive image frames. A shape-adaptive triangular patch mesh model overlay is provided on the first image frame such that the location of node points of each patch is determined by the spatial image gradients of the first frame and the pixel-to-pixel motion vectors. A priority ranking of patches is established before determining the node point motion vectors. The node point motion vectors, representing the motion of each of the node points of the triangular mesh patches, are estimated by a linear least-squares solution to an affine transformation of the mesh overlay on the first frame into the second frame. Failure regions are identified and are revised in accordance with a data bit budget. All data are coded prior to transmission at a constant data bit rate by a sending unit to a receiving unit over a data link.

Video compression system using a dense motion vector field and a

Growing community of inventors

Rochester, NY, United States of America

A Murat Tekalp