Method and apparatus for estimating scene structure and ego-motion from multiple images of a scene using correlation

Abstract

A system that estimates both the ego-motion of a camera through a scene and the structure of the scene by analyzing a batch of images of the scene obtained by the camera employs a correlation-based, iterative, multi-resolution algorithm. The system defines a global ego-motion constraint to refine estimates of inter-frame camera rotation and translation. It also uses local window-based correlation to refine the current estimate of scene structure. The batch of images is divided into a reference image and a group of inspection images. Each inspection image in the batch of images is aligned to the reference image by a warping transformation. The correlation is determined by analyzing respective Gaussian/Laplacian decompositions of the reference image and warped inspection images. The ego-motion constraint includes both rotation and translation parameters. These parameters are determined by globally correlating surfaces in the respective inspection images to the reference image. Scene structure is determined on a pixel-by-pixel basis by correlating multiple pixels in a support region among all of the images. The correlation surfaces are modeled as quadratic or other parametric surfaces to allow easy recognition and rejection of outliers and to simplify computation of incremental refinements for ego-motion and structure. The system can employ information from other sensors to provide an initial estimate of ego-motion and/or scene structure. The system operates using images captured by either single-camera rigs or multiple-camera rigs.