Lucent Technologies Inc.

Alcatel Lucent

Daniel G Aliaga

Ingrid Birgitta Carlbom

Thomas A Funkhouser

Dimah V Yanovsky

Ingrid B Carlborn

Techniques for computing error-bounded position and orientation (pose) of a panoramic camera in real-world environments. Such environments may include large interior spaces (e.g., buildings) A space may include multiple rooms. For example, a technique for capturing images associated with an environment includes the following steps/operations. First, respective placements of fiducials in the environment are determined so as to satisfy at least one constraint. Images are captured, with an image capture device (e.g., camera), associated with the environment with the fiducials placed therein. A pose estimation of the image capture device is then determined based on projections of the fiducials in the captured images. The pose estimation may be optimized so as to obtain an optimal pose per image. Also, the fiducial placements may be optimized so as to obtain optimal fiducial placements. Then at least one constraint may include a constraint associated with the number of visible fiducials, a constraint associated with a distance from a viewpoint to a fiducial, and/or a constraint associated with an angle subtended by pairs of fiducials.

Method and apparatus for computing error-bounded position and orientation of panoramic cameras in real-world environments

A paraboloidal catadioptric camera is calibrated by relaxing the assumption of an ideal system to account for perspective projection, radial distortion, and mirror misalignment occurring within the camera system. Calibration points, which are small and visually distinct objects, are distributed at fixed locations within an environment. Omnidirectional images are captured by the catadioptric camera at different locations of the environment. Data points are obtained by identifying the location of the calibration points in each captured image. An optimization algorithm best-fits the data points to a perspective camera model in order to derive parameters, which are used to calibrate the catadioptric camera.

Camera model and calibration procedure for omnidirectional paraboloidal catadioptric cameras

Techniques for computing a globally consistent set of image feature correspondences across a wide range of viewpoints suitable for interactive walkthroughs and visualizations. The inventive approach takes advantage of the redundancy inherent in a dense set of images captured in a plane (or in higher dimensions, e.g., images captured in a volume, images captured over time, etc). The technique may detect features in a set of source images and track the features to neighboring images. When features track to the same position in the same image, they are flagged as potential correspondences. Among the potential correspondences, the technique selects the maximal set using a greedy graph-labeling algorithm (e.g., best-first order). Only correspondences that produce a globally consistent labeling are selected. After globalization is done, a set of features common to a group of images can be quickly found and used to warp and combine the images to produce an interpolated novel view of the environment.

Method and apparatus for finding feature correspondences between images captured in real-world environments

Techniques for compressing and decompressing images captured from viewpoints throughout an n-dimensional space. As opposed to conventional approaches, these techniques exploit image coherence over the entire n-dimensional space, providing quick access to images along arbitrary contiguous viewpoint paths during computer graphics applications. This is accomplished by combining a hierarchical data structure with a compression methodology. Such techniques provide for generating and arranging reference images and residual images into a spatial image hierarchy such as a binary tree structure. Images and residuals may be encoded using intra-image compression. A captured image is extracted via a sequence of image warping and addition operations.

Method and apparatus for compressing and decompressing images captured from viewpoints throughout N-dimensional space

Techniques for creating interactive walkthroughs or visualizations of real-world environments. Such environments for which interactive walkthroughs may be created may comprise large interior spaces (e.g., buildings), wherein a space may comprise multiple rooms. The interactive walkthroughs may be photorealistic and rendered from arbitrary viewpoints at interactive rates. In one aspect of the invention, a technique for generating a visualization representative of an environment comprises the following steps. A dense set of image data is captured from the environment. The captured image data is compressed for storage. A portion of the compressed image data may then be decompressed so as to render a visualization representative of the environment from a selected novel viewpoint. Another portion of the image data may be prefetched and decompressed such that captured image data surrounding the selected novel viewpoint is available to be rendered. The visualization of the environment may then be rendered from the selected novel viewpoint.

Method and system for creating interactive walkthroughs of real-world environment from set of densely captured images

An omnidirectional video camera captures images of the environment while moving along several intersecting paths forming an irregular grid. These paths define the boundaries of a set of image loops within the environment. For arbitrary viewpoints within each image loop, a 4D plenoptic function may be reconstructed from the group of images captured at the loop boundary. For an observer viewpoint, a strip of pixels is extracted from an image in the loop in front of the observer and paired with a strip of pixels extracted from another image on the opposite side of the image loop. A new image is generated for an observer viewpoint by warping pairs of such strips of pixels according to the 4D plenoptic function, blending each pair, and then stitching the resulting strips of pixels together.

Growing community of inventors

Millington, NJ, United States of America

Daniel G Aliaga