Microsoft Technology Licensing, LLC

Kenneth Mitchell Jakubzak

Mathew Julian Lamb

Amy Mun Hong

Austin Seungmin Lee

Julia Schwarz

Nicholas Ferianc Kamuda

Anthony James Ambrus

Sophie Stellmach

Sheng Kai Tang

Adam Edwin Behringer

Joshua Kyle Neff

Jenny Kam

Michael Harley Notter

Keiichi Matsuda

Jouya Jadidian

Toby Leonard Sharp

Thomas Joseph Cashman

Jamie Shotton

Erroll William Wood

Alton Kwok

Jingjing Shen

Brent Michael Wilson

Matthew Julian Lamb

Examples are disclosed that relate to utilizing a facial tracking sensor for controlling computer-generated facial expressions. One example provides a method of controlling computer-generated facial expressions. The method comprises receiving a sensor value acquired via a facial tracking sensor and determining an interpolated value for the sensor value within a value range. The value range corresponding to a blendshape range for a facial expression. The method further comprises determining a blendshape mapping based at least upon the interpolated value, determining expression data based at least upon the blendshape mapping, and providing the expression data to a device.

Controlling computer-generated facial expressions

Systems and methods are provided for facilitating the presentation of expressive intent and other status information with messaging and other communication applications. The expressive intent is based on expressive effect data associated with the message recipients and/or message senders. The expressive intent can be conveyed through avatars and modified message content. The avatars convey gestures, emotions and other status information and the presentation of the avatars can be reactive to detected state information of the message recipient(s), message sender(s) and/or corresponding messaging device(s).

Modifying an avatar to reflect a user's expression in a messaging platform

Examples are disclosed that relate to hand gesture-based emojis. One example provides, on a display device, a method comprising receiving hand tracking data representing a pose of a hand in a coordinate system, based on the hand tracking data, recognizing a hand gesture, and identifying an emoji corresponding to the hand gesture. The method further comprises presenting the emoji on the display device, and sending an instruction to one or more other display devices to present the emoji.

Hand gesture-based emojis

In various examples there is an apparatus for aligning three-dimensional, 3D, representations of people. The apparatus comprises at least one processor and a memory storing instructions that, when executed by the at least one processor, perform a method comprising accessing a first 3D representation which is an instance of a parametric model of a person; accessing a second 3D representation which is a photoreal representation of the person; computing an alignment of the first and second 3D representations; and computing and storing a hologram from the aligned first and second 3D representations such that the hologram depicts parts of the person which are observed in only one of the first and second 3D representations; or controlling an avatar representing the person where the avatar depicts parts of the person which are observed in only one of the first and second 3D representations.

Alignment of 3D representations for hologram/avatar control

Examples are disclosed that relate to utilizing image sensor inputs from different devices having different perspectives in physical space to construct an avatar of a first user in a video stream. The avatar comprises a three-dimensional representation of at least a portion of a face of the first user texture mapped onto a three-dimensional body simulation that follows actual physical movement of the first user. The three-dimensional body simulation of the first user is generated based on image data received from an imaging device and image sensor data received from a head-mounted display device both associated with the first user. The three-dimensional representation of the face of the first user is generated based on the image data received from the imaging device. The resulting video stream is sent, via a communication network, to a display device associated with a second user.

Multiple device sensor input based avatar

Growing community of inventors

Lynnwood, WA, United States of America

Kenneth Mitchell Jakubzak