Microsoft Technology Licensing, LLC

Daniel Stephen Wilde

Thomas Joseph Cashman

Matthew Alastair Johnson

Tadas Baltrusaitis

Stephan Joachim Garbin

Marta Malgorzata Wilczkowiak

Toby Leonard Sharp

Erroll William Wood

Christopher Maurice Mei

Marek Adam Kowalski

Jingjing Shen

Virginia Estellers Casas

Charles Thomas Hewitt

Julien Pascal Christophe Valentin

Martin De La Gorce

Timothy James Hutton

Paul Malcolm McIlroy

Nikola Milosavljevic

Ivan Stojiljkovic

Xian Xiao

Keypoints are predicted in an image. Predictions are generated for each of the keypoints of an image as a 2D random variable, normally distributed with location (x, y) and standard deviation sigma. A neural network is trained to maximize a log-likelihood that samples from each of the predicted keypoints equal a ground truth. The trained neural network is used to predict keypoints of an image without generating a heatmap.

Model fitting using keypoint regression

There is a method of computing a stylized, animatable representation of a subject from a family of stylized animatable representations. The method comprises: accessing a realistic representation of the subject; computing a mesh mapping using a model that was formed using a data set of training examples which pair realistic representations of other subjects with instances of the family. The method also comprises applying the mesh mapping to the realistic representation of the subject to produce a target mesh; and selecting the stylized animatable representation from the family, by assessing closeness of the target mesh with instances of the family.

Stylized animatable representation

In various examples there is an apparatus for computing an image depicting a face of a wearer of a head mounted display (HMD), as if the wearer was not wearing the HMD. An input image depicts a partial view of the wearer's face captured from at least one face facing capture device in the HMD. A machine learning apparatus is available which has been trained to compute expression parameters from the input image. A 3D face model that has expressions parameters is accessible as well as a photorealiser being a machine learning model trained to map images rendered from the 3D face model to photorealistic images. The apparatus computes expression parameter values from the image using the machine learning apparatus. The apparatus drives the 3D face model with the expression parameter values to produce a 3D model of the face of the wearer and then renders the 3D model from a specified viewpoint to compute a rendered image. The rendered image is upgraded to a photorealistic image using the photorealiser.

Computing images of head mounted display wearer

A computer system is provided that includes an input system and a processor that may be configured to control a virtual manipulator based on input data received from the input system. The processor is further configured to determine an initial state of a system that includes at least an initial state of the virtual manipulator and an initial state of a virtual object, detect at least one contact point between a portion of the virtual manipulator and the virtual object. The processor is further configured to calculate a subsequent state for the virtual object that minimizes a set of energies or residuals defined in terms of the one or more positional quantities determined for the initial state of the system and the one or more positional quantities determined for the subsequent state of the system using a position-based energy minimizing function.

Position based energy minimizing function

Growing community of inventors

Cambridge, United Kingdom

Daniel Stephen Wilde