Microsoft Technology Licensing, LLC

Other

Padmananbhan Anandan

Richard S Szeliski

Simon John Baker

Zhengyou Zhang

Heung-Yeung Shum

Michal Irani

Philip H S Torr

Daphna Weinshall

Automated layer extraction from 2D images making up a 3D scene, and automated image pixel assignment to layers, to provide for scene modeling, is disclosed. In one embodiment, a computer-implemented method determines a number of planes, or layers, and assigns pixels to the planes. The method can determine the number of planes by first determining the high-entropy pixels of the images, and then determining a 1-plane through a predetermined n-plane estimation, such as via a robust estimation, and a most likely x-plane estimation, where x is between 1 and n, such as via a Bayesian approach. Furthermore, the method can assign pixels via an iterative EM approach based on classifying criteria.

Automated layer extraction and pixel assignment from image sequences

The present invention is embodied in systems and methods for determining structure and motion of a three-dimensional (3D) object using two-dimensional (2D) images of the object obtained from multiple sets of views with different projection models, such as from a full perspective view and a weak perspective views. A novel fundamental matrix is derived that embodies the epipolar geometry between a full perspective view and a weak perspective view. The systems and methods of the present invention preferably uses the derived fundamental matrix together with the 2D image information of the full and weak perspective views to digitally reconstruct the 3D object and produce results with multi-resolution processing techniques. These techniques include recovering and refining motion parameters and recovering and refining structure parameters of the fundamental matrix. The results can include, for example, 3D positions of points, camera position between different views, texture maps, and the like.

System and method for determining structure and motion from two-dimensional images for multi-resolution object modeling

A system and method for extracting structure from stereo that represents the scene as a collection of planar layers. Each layer optimally has an explicit 3D plane equation, a colored image with per-pixel opacity, and a per-pixel depth value relative to the plane. Initial estimates of the layers are recovered using techniques from parametric motion estimation. The combination of a global model (the plane) with a local correction to it (the per-pixel relative depth value) imposes enough local consistency to allow the recovery of shape in both textured and untextured regions.

Stereo reconstruction employing a layered approach

A system and method for extracting structure from stereo that represents the scene as a collection of planar layers. Each layer optimally has an explicit 3D plane equation, a colored image with per-pixel opacity, and a per-pixel depth value relative to the plane. Initial estimates of the layers are made and then refined using a re-synthesis step which takes into account both occlusions and mixed pixels. Reasoning about these effects allows the recovery of depth and color information with high accuracy, even in partially occluded regions. Moreover, the combination of a global model (the plane) with a local correction to it (the per-pixel relative depth value) imposes enough local consistency to allow the recovery of shape in both textured and untextured regions.

Growing community of inventors

Issaquah, WA, United States of America

Padmananbhan Anandan