Microsoft Technology Licensing, LLC

University of Illinois

Derek Edward Hoiem

Pushmeet Kohli

Martyn S Lovell

Craig Andrew Critchley

Steve Seixeiro

Eric Robert Berman

Victor A Stone

Timothy R Sullivan

Peter E Loforte

Kevin Karsch

Justin McRoberts

David Alexander Forsyth

Mani Golparvar-Fard

Edward Low Mills

Varsha Chandrashekhar Hedau

Jacob Je-Chian Lin

Joseph M Degol

Kook In Han

A system initializes a set of calibrated images with known 3D pose relative to a 3D building information model (BIM) to be anchor images, detects features within images of unknown position and orientation, and determines matches with features of the calibrated images. The system determines a subset of the images that have at least a threshold number of matching features, selects an image from the subset of the images having the largest number of matching features, and executes a reconstruction algorithm using the image and the anchor images to calibrate the image to the BIM and generate an initial 3D point cloud model. The system repeats the last steps to identify a second image from the subset and perform, starting with the initial 3D point cloud model and using the second image, 3D reconstruction to generate an updated 3D point cloud model that is displayable in a graphical user interface.

Computation of point clouds and joint display of point clouds and building information models with project schedules for monitoring construction progress, productivity, and risk for delays

An image into which one or more objects are to be inserted is obtained. Based on the image, both a 3-dimensional (3D) representation and a light model of the scene in the image are generated. One or more objects are added to the 3D representation of the scene. The 3D representation of the scene is rendered, based on the light model, to generate a modified image that is the obtained image modified to include the one or more objects.

Inserting objects into content

Surface segmentation from RGB and depth images is described. In one example, a computer receives an image of a scene. The image has pixels which each have an associated color value and an associated depth value representing a distance between from an image sensor to a surface in the scene. The computer uses the depth values to derive a set of three-dimensional planes present within the scene. A cost function is used to determine whether each pixel belongs to one of the planes, and the image elements are labeled accordingly. The cost function has terms dependent on the depth value of a pixel, and the color values of the pixels and at least one neighboring pixel. In various examples, the planes can be extended until they intersect to determine the extent of the scene, and pixels not belonging to a plane can be labeled as objects on the surfaces.

Surface segmentation from RGB and depth images

A unified user interface includes one or more component tables and a master table. The one or more component tables include resource information for the user interface with respect to a particular component. The master table includes resource information for the application and is merged from the one or more component tables. The component tables may be added or subtracted at any time and the master table is recreated by again merging the remaining or now existing component tables. The master table is used by a host application to build the user interface for a suite application. Components are only loaded and corresponding user interfaces built when appropriate commands are accessed. Thus, applications may be developed and components can be added or modified at a later time without rewriting the shell application or re-releasing a product.

Method and apparatus for a unified user interface

An application and a context-based help system operate on a computer. The application includes a plurality of components that effectuate application functions, where each component of the application is constructed to describe a current context of the component representative of a current state of the component. At least some of the components are active at any one time. The help system includes a help engine that receives the current context from each active component, dynamically builds a current overall context of the application based on each received current context, locates help topics based on the current overall context, and filters and prioritizes the located help topics.

Context-based help engine and dynamic help

An improved mechanism for converting handwritten input data into text and commands in a computer system is provided. An unintended results detection component is used after the handwritten data has been converted to text and commands. The unintended results detection component determines when the converted text and commands were likely unintended by the computer user. If the unintended results detection component determines that the converted text and commands were likely unintended by the computer user, the unintended results detection component outputs an indication to the user. The user then has the option of accepting or rejecting the interpretation of the system of the handwritten input data.

Unintended results detection in a pen-based computer system

Growing community of inventors

Sammamish, WA, United States of America

Derek Edward Hoiem