Microsoft Technology Licensing, LLC

Jiawen Chen

Shahram Izadi

Stephen Edward Hodges

Andrew William Fitzgibbon

David Alexander Butler

David Kim

Otmar Hilliges

Patrick Luke Olivier

Dennis Bautembach

Iason Oikonomidis

Scalable volumetric reconstruction is described whereby data from a mobile environment capture device is used to form a 3D model of a real-world environment. In various examples, a hierarchical structure is used to store the 3D model where the structure comprises a root level node, a plurality of interior level nodes and a plurality of leaf nodes, each of the nodes having an associated voxel grid representing a portion of the real world environment, the voxel grids being of finer resolution at the leaf nodes than at the root node. In various examples, parallel processing is used to enable captured data to be integrated into the 3D model and/or to enable images to be rendered from the 3D model. In an example, metadata is computed and stored in the hierarchical structure and used to enable space skipping and/or pruning of the hierarchical structure.

Scalable volumetric 3D reconstruction

A wearable sensor for tracking articulated body parts is described such as a wrist-worn device which enables 3D tracking of fingers and optionally also the arm and hand without the need to wear a glove or markers on the hand. In an embodiment a camera captures images of an articulated part of a body of a wearer of the device and an articulated model of the body part is tracked in real time to enable gesture-based control of a separate computing device such as a smart phone, laptop computer or other computing device. In examples the device has a structured illumination source and a diffuse illumination source for illuminating the articulated body part. In some examples an inertial measurement unit is also included in the sensor to enable tracking of the arm and hand

Wearable sensor for tracking articulated body-parts

Methods of animating objects using the human body are described. In an embodiment, a deformation graph is generated from a mesh which describes the object. Tracked skeleton data is received which is generated from sensor data and the tracked skeleton is then embedded in the graph. Subsequent motion which is captured by the sensor result in motion of the tracked skeleton and this motion is used to define transformations on the deformation graph. The transformations are then applied to the mesh to generate an animation of the object which corresponds to the captured motion. In various examples, the mesh is generated by scanning an object and the deformation graph is generated using orientation-aware sampling such that nodes can be placed close together within the deformation graph where there are sharp corners or other features with high curvature in the object.

Animating objects using the human body

Growing community of inventors

Cambridge, United Kingdom

Jiawen Chen