Direct method for modeling non-rigid motion with thin plate spline transformation

Abstract

A system and a method model the motion of a non-rigid object using a thin plate spline (TPS) transform. A first image of a video sequence is received, and a region of interest, referred to as a template, is chosen manually or automatically. A set of arbitrarily-chosen fixed reference points is positioned on the template. A target image of the video sequence is chosen for motion estimation relative to the template. A set of pixels in the target image corresponding to the pixels of the template is determined, and this set of pixels is back-warped to match the template using a thin-plate-spline-based technique. The error between the template and the back-warped image is determined and iteratively minimized using a gradient descent technique. The TPS parameters can then be used to estimate the relative motion between the template and the corresponding region of the target image. According to one embodiment, a stiff-to-flexible approach mitigates instability that can arise when reference points lie in textureless regions, or when the initial TPS parameters are not close to the desired ones. The value of a regularization parameter is varied from a larger to a smaller value, varying the nature of the warp from stiff to flexible, so as to progressively emphasize local non-rigid deformations.