Segmenting encoding system encoding video data using segment-by-segment kinetic data including change information in addition to translation information

Abstract

Video data is compressed with nonkey frames encoded with reference to segmentation of reference frames, where the encoded video data includes kinetic information relating segments of a reference frame to pixels of a nonkey frame and the kinetic information includes translations of segments and at least one of a z-order, a deformation and a lighting change. The segmentation performed during encoding can be included in whole or part in the compressed video data. If used, z-ordering could be relative or absolute, based on changes of occlusion of segments by other segments between the frames, based on content of other frames or based on z-order indications in the video data being compressed. Other kinetic information might include segment changes between frames such as rotation, dilation, affine transformations, nonlinear transformations defined by a set of deformation parameters, linear lighting offsets in one, two or three color planes, and/or residue information.