University of Illinois

Adobe, Inc.

Microsoft Technology Licensing, LLC

Lumenco, LLC

Kevin Karsch

David Alexander Forsyth

Hailin Jin

Nathan Aaron Carr

Kalyan Krishna Sunkavalli

Sunil Sharadchandra Hadap

Sing Bing Kang

Mark A Raymond

Ce Liu

Hector Andres Porras Soto

Derek Edward Hoiem

Zicheng Liao

Mani Golparvar-Fard

Varsha Chandrashekhar Hedau

A system and method are disclosed for, using structure-from-motion techniques, projecting a building information model (BIM) into images from photographs taken of a construction site, to generate a 3D point cloud model using the BIM and, when combined with scheduling constraints, facilitates 4D visualizations and progress monitoring. One of the images acts as an anchor image. Indications are received of first points in the anchor image that correspond to second points in the BIM. Calibration information for an anchor camera is calculated based on the indications and on metadata extracted from the anchor image, to register the anchor image in relation to the BIM. A homography transformation is determined between the images and the anchor camera using the calibration information, to register the rest of the images with the BIM, where some of those images are taken from different cameras and from different angles to the construction site.

4D vizualization of building design and construction modeling with photographs

A method for generating additional views from a stereo image defined by a left eye image and a right eye image. The method includes receiving as input at least one stereo image. The method includes, for each of the stereo images, generating a plurality of additional images. The method includes interlacing the additional images for each of the stereo images to generate three dimensional (3D) content made up of multiple views of the scenes presented by each of the stereo images. The interlacing may be performed such that the generated 3D content is displayable on a 3D display device including a barrier grid or a lenticular lens array on the monitor screen. The additional images may include 12 to 40 or more frames providing views of the one or more scenes from differing viewing angles than provided by the left and right cameras used to generate the original stereo image.

Conversion of a digital stereo image into multiple views with parallax for 3D viewing without glasses

Image editing techniques are disclosed that support a number of physically-based image editing tasks, including object insertion and relighting. The techniques can be implemented, for example in an image editing application that is executable on a computing system. In one such embodiment, the editing application is configured to compute a scene from a single image, by automatically estimating dense depth and diffuse reflectance, which respectively form the geometry and surface materials of the scene. Sources of illumination are then inferred, conditioned on the estimated scene geometry and surface materials and without any user input, to form a complete 3D physical scene model corresponding to the image. The scene model may include estimates of the geometry, illumination, and material properties represented in the scene, and various camera parameters. Using this scene model, objects can be readily inserted and composited into the input image with realistic lighting, shadowing, and perspective.

Automatic geometry and lighting inference for realistic image editing

A fragment is relit for insertion into a target scene of an image by obtaining a fragment model for the fragment. A set of detail maps for the fragment model are generated, each of which encodes fine-scale shading effects from the surface detail of the fragment. A target scene model is obtained for the target scene, and the fragment model is inserted into the target scene model. The target scene model with inserted fragment model is rendered, and a composited target scene is generated. A modified target scene is generated by combining the composited target scene and the set of detail maps. Weights assigned to the different detail maps can be changed by the user, allowing the modified target scene to be readily altered without re-rendering the target scene model with the inserted fragment model.

Relighting fragments for insertion into content

In general, a 'Stereoscopic Video Converter' (SVC) provides various techniques for automatically converting arbitrary 2D video sequences into perceptually plausible stereoscopic or '3D' versions while optionally generating dense depth maps for every frame of the video sequence. In particular, the automated 2D-to-3D conversion process first automatically estimates scene depth for each frame of an input video sequence via a label transfer process that matches features extracted from those frames with features from a database of images and videos having known ground truth depths. The estimated depth distributions for all image frames of the input video sequence are then used by the SVC for automatically generating a &#x201c;right view&#x201d; of a corresponding stereoscopic image for each frame (assuming that each original input frame represents the &#x201c;left view&#x201d; of the stereoscopic image).

Automatic 2D-to-stereoscopic video conversion

An image into which one or more objects are to be inserted is obtained. Based on the image, both a 3-dimensional (3D) representation and a light model of the scene in the image are generated. One or more objects are added to the 3D representation of the scene. The 3D representation of the scene is rendered, based on the light model, to generate a modified image that is the obtained image modified to include the one or more objects.

Inserting objects into content

Growing community of inventors

Urbana, IL, United States of America

Kevin Karsch