Autodesk, Inc.

Bentley Systems, Inc.

Inria Institut National De Recherche En Informatique Et En Automatique

Luc Robert

Jean-Philippe Pons

Emmanuel Gallo

Elenie Godzaridis

Stephane Nullans

Nicolas Gros

George William Fitzmaurice

Todd Christopher Nicol

Cyril Novel

James La Fleur

Prince Ngattai Lam

Lucas Malleus

Alexandrina Orzan

Frederic Precioso

Diane Lingrand

Yann Noutary

Hugo Lavezac

Kimberly Whinna Cottrell

Barbara Balents

Gr&eacute;goire Malandain

Stéphane Nullans

A sample may be generated for each point of a plurality of point clouds that represent a scene. A visibility ray may be created between each point of the plurality of point clouds and the one or more sources that generated the point. One or more sample, if any, that intersect a visibility ray may be identified. Each point corresponding to an intersecting sample may be determined to represent or likely represent an unwanted object if the visibility ray is from a different source that did not generate the point and the point is not coherent with any points generated by the different source. A visibility score for each point determined to represent or likely represent an unwanted object may be adjusted. A model may be generated, wherein the model does not include the unwanted object in the scene but includes the permanent object with see-through characteristics in the scene.

Systems, methods, and media for filtering points of a point cloud utilizing visibility factors to generate a model of a scene

Techniques are provided for modifying coloring of images utilizing machine learning. A trained model is generated utilizing machine learning with training data that includes images of a plurality of different scenes with different illumination characteristics. New original images of a scene may each be downsampled and transformed to a corresponding output image utilizing the trained model. A color transformation from each original image to its corresponding output image may be determined. In an embodiment, the color transformation is determined utilizing a spline fitting approach. The determined color transformations may be applied to each of the original images to generate corrected images. Specifically, the color transformation that is applied to a particular original image is the color transformation determined for the input image that corresponds to the particular original image. The corrected images are utilized to generate a digital model of the scene, and the digital model has accurate model texture.

Systems, methods, and media for modifying the coloring of images utilizing machine learning

In example embodiments, techniques are provided for calculating camera rotation using translations between sensor-derived camera positions (e.g., from GPS) and pairwise information, producing a sensor-derived camera pose that may be integrated in an early stage of SfM reconstruction. A software process of a photogrammetry application may obtain metadata including sensor-derived camera positions for a plurality of cameras for a set of images and determine optical centers based thereupon. The software process may estimate unit vectors along epipoles from a given camera of the plurality of cameras to two or more other cameras. The software process then may determine a camera rotation that best maps unit vectors defined based on differences in the optical centers to the unit vectors along the epipoles. The determined camera rotation and the sensor-derived camera position form a sensor-derived camera pose that may be returned and used.

Determining camera rotations based on known translations

In an example embodiment, techniques are provided for locking a region of fully-connected large-scale multi-dimensional spatial data (e.g., a large-scale 3-D mesh) defined by a bounding box. A region is associated with a lock state (e.g., exclusive or sharable). Clients may access the fully-connected large-scale multi-dimensional spatial data based on a comparison of the bounding box of the requested spatial data to the bounding boxes of other client's locks and their lock state.

Locking of spatial regions of large-scale fully-connected multi-dimensional spatial data for collaborative updating

In an example embodiment, techniques are provided for concurrently editing fully-connected large-scale multi-dimensional spatial data (e.g., a large-scale 3-D mesh) by ensuring that edits performed by multiple clients are non-conflicting edits that are 'trivially' mergeable (e.g. mergeable simply via cut-and-paste operations). Conflicting edits may be prevented by locks (e.g., region-based locks). Non-conflicting edits that require 'non-trivial' merging may be prevented through the use of marked read-only boundaries.

Techniques for concurrently editing fully connected large-scale multi-dimensional spatial data

Methods, systems, and apparatus including medium-encoded computer program products for generating and visualizing 3D scenes include, in one aspect, a method including: obtaining site data acquired by one or more capture devices, wherein the site data comprises data sets corresponding to two or more locations about a physical site, and each respective data set comprises (i) imaging data of the physical site, (ii) coordinate data for the imaging data, and (iii) time metadata for the imaging data; reconstructing a series of three dimensional (3D) modeled scenes of the physical site from the site data using the imaging data, the coordinate data, and the time metadata; receiving a request having associated position, orientation and time data; and generating, in response to the request, output for display of a portion of the 3D modeled scenes to represent the physical site based on the position, orientation and time data.

Systems and methods for generating time discrete 3D scenes

A method, system, apparatus, article of manufacture, and computer program product provide the ability to detect junctions. 3D pixel image data is obtained/acquired based on 2D image data and depth data. Within a given window over the 3D pixel image data, for each of the pixels within the window, an equation for a plane passing through the pixel is determined/computed. For all of the determined planes within the given window, an intersection of all of the planes is computed. A spectrum of the intersection/matrix is analyzed. Based on the spectrum, a determination is made if the pixel at the intersection is of 3 or more surfaces, 2 surfaces, or is 1 surface.

Combining two-dimensional images with depth data to detect junctions or edges

A method, system, and apparatus provide the ability to globally register point cloud scans. A first and a second three-dimensional (3D) point cloud are acquired. The point clouds have a subset of points in common and there is no prior knowledge on an alignment between the point clouds. Particular points that are likely to be identified in the other point cloud are detected. Information about a normal of each of the detected particular points is retrieved. A descriptor (that only describes 3D information) is built on each of the detected particular points. Matching pairs of descriptors are determined. Rigid transformation hypotheses are estimated (based on the matching pairs) and represent a transformation. The hypotheses are accumulated into a fitted space, selected based on density, and validated based on a scoring. One of the hypotheses is then selected as a registration.

Keypoint-based point-pair-feature for scalable automatic global registration of large RGB-D scans

Methods, systems, and apparatus, including medium-encoded computer program products, for generating and visualizing 3D scenes include, in one aspect, a method including: obtaining metadata for 2D images that are related to a location of interest; searching for discrete image correspondences between pairs of the 2D images; grouping the 2D images into different time periods, including inferring one or more time periods for a portion of the 2D images that do not have date metadata; receiving a selection of at least one of the different time periods; finding camera intrinsic and extrinsic parameters for each image grouped into the selected time period; reconstructing a 3D scene of the location during the selected time period from the 2D images grouped into the selected time period using the camera intrinsic and extrinsic parameters; and providing the 3D scene for use in displaying the location of interest from different 3D perspectives.

Systems and methods for generating 3D scenes with time element for display

A method, system, and computer program product provide the ability to dynamically generate a three-dimensional (3D) scene. A red green blue (RGB) image (in RGB color space) of the 3D scene is acquired. The RGB image is converted from RGB color space to a luminance (Y) and chrominance (UV) image in YUV color space (hat includes Y information and UV information). Reflectance information of the 3D scene is acquired from a laser scanner. Based on a blending function, the luminance information is blended with the reflectance information resulting in a blended YUV image. The blended YUV image is converted from YUV color space into RGB color space resulting in a blended RGB image that is output.

Dynamically creating and presenting a three-dimensional (3D) view of a scene by combining color, brightness, and intensity from multiple scan data sources

A system, apparatus, method, computer program product, and computer readable storage medium provide the ability to reconstruct a surface mesh. Photo image data is obtained from a set of overlapping photographic images. Scan data is obtained from a scanner. A point cloud is generated from a combination of the photo image data and the scan data. An initial rough mesh is estimated from the point cloud data. The initial rough mesh is iteratively refined into a refined mesh.

Mesh reconstruction from heterogeneous sources of data

A method, apparatus, system, article of manufacture, and computer readable storage medium provides the ability to determine two or more camera viewpoint optical centers. A first image and a second image captured by camera devices (and the rotations for the camera devices) are obtained. For each pair of matched points between the first image and the second image, a linear equation is defined that utilizes the rotations, pixel coordinates of the matched points and optical centers. A matrix AA is computed where each row of A corresponds to a pair of matched points and the number of columns is 3N (N representing the number of camera devices). The matrix is solved resulting in an eigenvector that holds the optical centers that are used to output image information.

Camera translation using rotation from device

The invention relates to the field of binary image processing operations, such as thinning or thickening of 2D or 3D images with pixels or with voxels, respectively. The device according to the invention has processing means which rely on the logic of binary decision diagrams, of reduced order and with inverting branch instructions, in particular for determining whether or not a current element is a simple point. The binary decision diagram then defines a set of comparisons of bits of the elements of a chosen neighbourhood of the current element, which ends up at a terminal node &ldquo;1&rdquo; or a terminal node &ldquo;0&rdquo;, defining a particular property of the current element.

Growing community of inventors

Valbonne, France

Luc Robert