Disney Enterprises, Inc.

Eth Zurich

Microsoft Technology Licensing, LLC

Honda Motor Co., Ltd.

The University of North Carolina at Chapel Hill

Hexagon Technology Center Gmbh

Marc Pollefeys

Oliver Wang

Alexander Sorkine Hornung

Benjamin Resch

Hendrik Lensch

Markus Gross

Szymon Piotr Stachniak

Tunc Ozan Aydin

Paul A Beardsley

Gregory Charles Walsh

Aljosa A Smolic

Jan-Michael Frahm

Jongwoo Lim

William Douglas Guyman

Michelle Brook

Hendrik Mark Langerak

Yagiz Aksoy

Bernhard Zeisl

Silvano Galliani

Kevin Köser

Jereon Van Baar

The described implementations relate to images and depth information and generating useful information from the images and depth information. One example can identify planes in a semantically-labeled 3D voxel representation of a scene. The example can infer missing information by extending planes associated with structural elements of the scene. The example can also generate a watertight manifold representation of the scene at least in part from the inferred missing information.

Deriving information from images

A method of determining a registration between Scanworlds may include determining a first viewpoint of a setting based on first point data of a first Scanworld. The first Scanworld may include information about the setting as taken by a first laser scanner at a first location. The method may further include determining a second viewpoint of the setting based on second point data of a second Scanworld. The second Scanworld may include information about the setting as taken by a second laser scanner at a second location. The method may further include generating a first rectified image based on the first viewpoint and generating a second rectified image based on the second viewpoint. Additionally, the method may include determining a registration between the first Scanworld and the second Scanworld based on the first viewpoint and the second viewpoint.

Registration of multiple laser scans

Scenes reconstruction may be performed using videos that capture the scenes at high resolution and frame rate. Scene reconstruction may be associated with determining camera orientation and/or location ('camera pose') throughout the video, three-dimensional coordinates of feature points detected in frames of the video, and/or other information. Individual videos may have multiple frames. Feature points may be detected in, and tracked over, the frames. Estimations of camera pose may be made for individual subsets of frames. One or more estimations of camera pose may be determined as fixed estimations. The estimated camera poses for the frames included in the subsets of frames may be updated based on the fixed estimations. Camera pose for frames not included in the subsets of frames may be determined to provide globally consistent camera poses and three-dimensional coordinates for feature points of the video.

Systems and methods for facilitating three-dimensional reconstruction of scenes from videos

This disclosure related to systems and methods of palette-based color editing. A set of principle colors may be identified from among individual colors of individual pixels of an image. Color distributions about the individual principle colors may be determined. An individual color distribution may include a homogenous range of colors that may vary with respect to an individual principle color. Pixels may be associated with one or more pixel groups based on correspondences between individual colors of the individual pixels and individual colors included in the individual color distributions corresponding to the individual pixel groups. Color editing may be effectuated by modifying colors of pixels in a given group independently from other colors of other pixels in other groups.

System and method facilitating palette-based color editing

To generate a pixel-accurate depth map, data from a range-estimation sensor (e.g., a time-of flight sensor) is combined with data from multiple cameras to produce a high-quality depth measurement for pixels in an image. To do so, a depth measurement system may use a plurality of cameras mounted on a support structure to perform a depth hypothesis technique to generate a first depth-support value. Furthermore, the apparatus may include a range-estimation sensor which generates a second depth-support value. In addition, the system may project a 3D point onto the auxiliary cameras and compare the color of the associated pixel in the auxiliary camera with the color of the pixel in reference camera to generate a third depth-support value. The system may combine these support values for each pixel in an image to determine respective depth values. Using these values, the system may generate a depth map for the image.

Sensor fusion for depth estimation

A system and method are disclosed for online mapping of large-scale environments using a hybrid representation of a metric Euclidean environment map and a topological map. The system includes a scene module, a location recognition module, a local adjustment module and a global adjustment module. The scene flow module is for detecting and tracking video features of the frames of an input video sequence. The scene flow module is also configured to identify multiple keyframes of the input video sequence and add the identified keyframes into an initial environment map of the input video sequence. The location recognition module is for detecting loop closures in the environment map. The local adjustment module enforces local metric properties of the keyframes in the environment map, and the global adjustment module is for optimizing the entire environment map subject to global metric properties of the keyframes in the keyframe pose graph.

Online environment mapping

Growing community of inventors

Zurich, Switzerland

Marc Pollefeys