Microsoft Technology Licensing, LLC

Julien Pascal Christophe Valentin

Jingjing Shen

Matthew Alastair Johnson

Erik Alexander Hill

Erroll William Wood

Tadas Baltrusaitis

Stephan Joachim Garbin

Marek Adam Kowalski

Toby Leonard Sharp

Thomas Joseph Cashman

Virginia Estellers Casas

Charles Thomas Hewitt

Ivan Razumenic

Martin De La Gorce

Daniel Stephen Wilde

Predrag Rodoljub Jovanovic

Nikola Milosavljevic

Ivan Stojiljkovic

Shideh Rezaeifar

Kacper Kania

Stanislaw Kacper Szymanowicz

Keypoints are predicted in an image. Predictions are generated for each of the keypoints of an image as a 2D random variable, normally distributed with location (x, y) and standard deviation sigma. A neural network is trained to maximize a log-likelihood that samples from each of the predicted keypoints equal a ground truth. The trained neural network is used to predict keypoints of an image without generating a heatmap.

Model fitting using keypoint regression

A cage of primitive 3D elements and associated animation data is received. Compute a ray from a virtual camera through a pixel into the cage animated according to the animation data and compute a plurality of samples on the ray. Compute a transformation of the samples into a canonical cage. For each transformed sample, query a plurality of learnt radiance field parameterizations, each learnt on a different deformed state of the 3D scene to obtain color values for each learnt radiance field. For each transformed sample, query a learnt radiance field parameterization of the 3D scene to obtain an opacity value. Compute, for each transformed sample, a weighted combination of the color values, wherein the weights are related to the local features. A volume rendering method is applied to the weighted combinations of the color and the opacity values producing a pixel value.

Dynamic 3D scene generation

The techniques described herein disclose a system that is configured to detect and track the three-dimensional pose of an object (e.g., a head-mounted display device) in a color image using an accessible three-dimensional model of the object. The system uses the three-dimensional pose of the object to repair pixel depth values associated with a region (e.g., a surface) of the object that is composed of material that absorbs light emitted by a time-of-flight depth sensor to determine depth. Consequently, a color-depth image (e.g., a Red-Green-Blue-Depth image or RGB-D image) can be produced that does not include dark holes on and around the region of the object that is composed of material that absorbs light emitted by the time-of-flight depth sensor.

Repairing image depth values for an object with a light absorbing surface

To compute an image of a dynamic 3D scene comprising a 3D object, a description of a deformation of the 3D object is received, the description comprising a cage of primitive 3D elements and associated animation data from a physics engine or an articulated object model. For a pixel of the image the method computes a ray from a virtual camera through the pixel into the cage animated according to the animation data and computes a plurality of samples on the ray. Each sample is a 3D position and view direction in one of the 3D elements. The method computes a transformation of the samples into a canonical cage. For each transformed sample, the method queries a learnt radiance field parameterization of the 3D scene to obtain a color value and an opacity value. A volume rendering method is applied to the color and opacity values producing a pixel value of the image.

Computing images of controllable dynamic scenes

A system receives input from a user to initiate a process of generating a holodouble of the user. The system obtains image data of the user and deconstructs the image data to obtain a set of sparse data that identifies one or more attributes associated with the image data the user. The system uses a holodouble training model to generate and train the holodouble of the user based on the set of sparse data and obtained image data. The system renders a representation of the holodouble to the user concurrently while capturing new image data of the user, receives input from the user comprising approval of the holodouble, and completes training of the holodouble by saving the holodouble for subsequent use. The subsequent use includes one or more remote visual communication sessions.

Holodouble: systems and methods for low-bandwidth and high-quality remote visual communication

Holodouble: systems and methods for low-bandwidth and high quality remote visual communication

Growing community of inventors

Zurich, Switzerland

Julien Pascal Christophe Valentin