Microsoft Technology Licensing, LLC

Apple Inc.

Ming Chuang

Alvaro Collet Romea

Kristofer Noel Iverson

Emmett Walter Lalish

Kyle L Simek

Maxime Meilland

Pedro Antonio Pinies Rodriguez

Steve Sullivan

Baptiste Angles

Pat Sweeney

Andrew Predoehl

Hugues Herve Hoppe

Zheng Wang

Khai Tran

Dirk Schroeder

Don Gillett

William Crow

Patrick John Sweeney

Fabian Andres Prada Nino

Yu Ping Lin

Dave B Lindsay

Steven Craig Sullivan

Dennis Evseev

Mengxin Li

Emmett Lalish

Various implementations disclosed herein include devices, systems, and methods that provide realistic acoustic simulation for sound from a virtual source based on a site specific 3D acoustic model of a physical environment. Some implementations obtain first sensor data representing at least a portion of a first boundary of a physical environment while second sensor data corresponding to a second boundary of the physical environment is absent from the first sensor data. In some implementations, the second boundary of the physical environment is estimated based on the first sensor data. Then, a 3D acoustic model representing the physical environment is determined, where the 3D acoustic model includes a 3D shape formed of at least the first boundary and the second boundary. In some implementations, acoustics are provided for sound from a virtual source corresponding to a position within the physical environment, where the acoustics are provided based the 3D acoustic model.

Simplified meshes for acoustics

Various implementations disclosed herein generate a mesh representing the surfaces in a physical environment. The mesh is generated using multi-resolution voxels based on detected depth information, e.g., from a depth camera. The techniques may use multiple hash tables to store the multi-resolution voxel data. For example, the hash tables may store each voxel's 3D position and a truncated signed distance field (TSDF) value corresponding to each voxels' distance to a nearest surface. Each of the multiple hash tables may include data corresponding to a different level of resolution and those resolutions may depend upon distance/noise or other factors. For example, voxels close to a depth camera may have a finer resolution and smaller size compared to voxels that are further from the depth camera. Techniques disclosed herein may involve using a meshing algorithm that combines multi-resolution voxel information stored in multiple hash tables to generate a single mesh.

Multi-resolution voxel meshing

Various implementations disclosed herein include devices, systems, and methods that refine a first 3D surface representation (e.g., a 3D point cloud or a 3D mesh) using a second 3D surface representation that includes a 3D geometric primitive. In some implementations, a first 3D surface representation of a physical environment is obtained including points at 3D locations determined based on data generated by a first sensor. In some implementations, a second 3D surface representation corresponding to at least a portion of the physical environment is obtained that includes at least one 3D geometric primitive. In some implementations, a determination whether to adjust the 3D locations of at least one point of the points of the first 3D surface representation is made based on the 3D geometric primitive, and the 3D locations of the at least one point is adjusted to align with the geometric primitive based on the determination.

3D surface representation refinement

Systems, devices, and methods are described herein for geometrically simplifying three-dimensional (3D) video data. In one aspect, a method may include obtaining 3D data, with the 3D data including a plurality of portions associated with a default resolution priority. A higher resolution priority may be associated with one or more portions of the 3D data. Next, portions of the 3D data may be sorted according to resolution priorities associated with each portion, and geometric simplification may be performed on the sorted portions of the 3D data, beginning with portions associated with a least resolution priority and continuing with portions associated with successively higher resolution priorities. The simplified 3D data may be processed, for example, for rendering on a computing device or transmission to another device for display or generation, such as a 3D printing device for generating a 3D object.

Adaptive decimation using 3D video features

Systems, devices, and methods are described herein for transforming three dimensional (3D) video data into a 3D printable model. In one aspect, a method for transforming 3D video data may include receiving 3D video data indicated or selected for 3D printing. The selected portion or 3D video data, which may include a frame of the 3D video data, may be repaired or modified to generate a 3D model that define at least one enclosed volume. At least one of the enclosed volumes of the 3D video data may be re-oriented based on at least one capability of a target 3D printing device. In some aspects, the re-orienting may be performed to optimize at least one of a total print volume or print orientation of the at least one enclosed volume. In some aspects, the method may be performed in response to a single selection or action performed by a user.

3D printing using 3D video data

In some examples, a computing device can determine synthetic meshes based on source meshes of a source mesh sequence and target meshes of a target mesh sequence. The computing device can then place the respective synthetic meshes based at least in part on a rigid transformation to define a processor-generated character. For example, the computing device can determine subsets of the mesh sequences based on a similarity criterion. The computing device can determine modified first and second meshes having a connectivity corresponding to a reference mesh. The computing device can then determine the synthetic meshes based on the modified first and second meshes. In some examples, the computing device can project source and target textures onto the synthetic mesh to provide projected source and target textures. The computing device can determine a synthetic texture registered to the synthetic mesh based on the projected source and target textures.

Motion capture and character synthesis

A computing system is configured for context-adaptive allocation of render model resources that may sacrifice some level of detail in a computational description of a 3D scene before rendering in order to accommodate resource limitations in a rendering environment such as available processor cycles, and/or bandwidth for data transmission to a processor. Such resource limitations can often preclude rendering a richly detailed 3D scene, particularly in full-motion and/or in real time. An importance function describing the relative perceptual importance of elements that make up the 3D scene is utilized to enable resources to be adaptively allocated so that more resources go to visual elements of the 3D scene that have a higher perceptual importance. The rendered output may thus optimize visual fidelity for the computational description within the resource constrained rendering environment.

Context-adaptive allocation of render model resources

Concepts and technologies are described herein for providing raster-based mesh decimation. Generally described, input data defining a model is processed to render the model as a depth map from a multitude of perspectives. By capturing depth map data from a multitude of perspectives, components of the model that are visible from the multitude of perspectives are captured in the depth map data and components that are blocked by the visible components are not captured in the depth map data. Point cloud data is generated by merging the depth map data captured from the multitude of perspectives. Output mesh data is generated by applying a surface reconstruction to the point cloud data. One or more mesh decimation algorithms may be applied to the mesh data. Image data may also be captured from the input data and applied to the output mesh data.

Raster-based mesh decimation

Consistent tessellation via topology-aware surface tracking is provided in which a series of meshes is approximated by taking one or more meshes from the series and calculating a transformation field to transform the keyframe mesh into each mesh of the series, and substituting the transformed keyframe meshes for the original meshes. The keyframe mesh may be selected advisedly based upon a scoring metric. An error measurement on the transformed keyframe exceeding tolerance or threshold may suggest another keyframe be selected for one or more frames in the series. The sequence of frames may be divided into a number of subsequences to permit parallel processing, including two or more recursive levels of keyframe substitution. The transformed keyframe meshes achieve more consistent tessellation of the object across the series.

Consistent tessellation via topology-aware surface tracking

Growing community of inventors

Bellevue, WA, United States of America

Ming Chuang