Microsoft Technology Licensing, LLC

Darren Alexander Bennett

Andrew Jackson Klein

Charla Marie Pereira

Enrico William Guld

Dean Alan Wadsworth

David J W Seymour

Kin Hang Chu

Jonathon Burnham Cobb

Helen Joan Hem Lam

Stephen G Latta

Kevin Geisner

Relja Markovic

Gregory Nelson Snook

Kudo Tsunoda

Julia Faye Taylor-Hell

You-Da Yang

Robert István Butterworth

Kathryn Stone Perez

Ryan M Geiss

Mark James Finocchio

Alex A Kipman

Kathleen Patricia Mulcahy

R Stephen Polzin

Jeremy Bruce Kersey

Daniel Lee Osborn

Ethan Harris Arnowitz

Evan Lewis Jones

Kyle Mouritsen

Kevin Thomas Mather

Victor Jia

Heather Joanne Alekson

William Axel Olsen

Cory Ryan Bramall

Michael Edward Harnisch

Jason A Carter

Stephen Michael Lucas

Justin Thomas Savino

Ryan Nadel

William Robert Schnurr

Sean Robert Puller

Tsz Fung Wan

Struan Andrew Robertson

Weihua Huang

Alexandre Pierre Michel Godin

Arni Mar Thrastarson

Cydney Brooke Nielsen

Jason Drew Vantomme

Joel Jamon Rendon

Kjartan Olafsson

Mahesh Keshav Kamat

Maya Alethea Miller-Vedam

Adam Roy Mitchell

Zachary D Wiesnoski

Various methods and systems are provided for authoring and presenting 3D presentations. Generally, an augmented or virtual reality device for each author, presenter and audience member includes 3D presentation software. During authoring mode, one or more authors can use 3D and/or 2D interfaces to generate a 3D presentation that choreographs behaviors of 3D assets into scenes and beats. During presentation mode, the 3D presentation is loaded in each user device, and 3D images of the 3D assets and corresponding asset behaviors are rendered among the user devices in a coordinated manner. As such, one or more presenters can navigate the scenes and beats of the 3D presentation to deliver the 3D presentation to one or more audience members wearing augmented reality headsets.

Authoring and presenting 3D presentations in augmented reality

Examples of augmented reality (AR) environment control advantageously employ multi-factor intention determination and include: performing a multi-factor intention determination for summoning a control object (e.g., a menu, a keyboard, or an input panel) using a set of indications in an AR environment, the set of indications comprising a plurality of indications (e.g., two or more of a palm-facing gesture, an eye gaze, a head gaze, and a finger position simultaneously); and based on at least the set of indications indicating a summoning request by a user, displaying the control object in a position proximate to the user in the AR environment (e.g., docked to a hand of the user). Some examples continue displaying the control object while at least one indication remains, and continue displaying the control object during a timer period if one of the indications is lost.

Multi-factor intention determination for augmented reality (AR) environment control

The disclosed technology is generally directed to mixed-reality devices. In one example of the technology, a first mixed-reality guide is provided mixed-reality devices, enabling the mixed-reality devices to operate the first mixed-reality guide while providing a mixed-reality view, such that: while the first mixed-reality guide is navigated to a step of the set of steps of the first mixed-reality guide, the mixed-reality view includes a hologram at a real-world location in the real-world environment at which work associated with the step is to be performed. From each mixed-reality device, mixed-reality data is received based on use of at least the first mixed-reality guide on the mixed-reality device. The mixed-reality data includes spatial telemetry data collected for at least one step of the first mixed-reality guide. A presentation that is based on the mixed-reality data is provided. The first mixed-reality guide is enabled to be altered based on the mixed-reality data.

Mixed-reality guide data collection and presentation

Various methods and systems are provided for puppeteering in augmented reality. Generally, an augmented or virtual reality device for each user generates a virtual 3D environment comprising a virtual representation of a physical room and a 3D asset. An author can record a 3D path for a puppeteering animation of the 3D asset using a 3D interface. At the same time, a coordinated rendering of a corresponding 3D image moving along the 3D path is updated among author devices substantially in real-time. Distinct states of the 3D asset can be assigned to different portions of the 3D path, and authors can set behavior parameters to assign template behaviors such as obstacle avoidance, particle effects, path visualizations and physical effects. The behavior parameters are distributed among presenter and audience devices, and a coordinated rendering of an animation of the 3D image corresponding to the puppeteering animation is triggered.

Puppeteering in augmented reality

The disclosed technology is generally directed to mixed reality devices. In one example of the technology, a mixed-reality view is caused to be provided to an operator, wherein the mixed-reality view includes both a real-world environment of the operator and holographic aspects. The operator is enabled to navigate among a plurality of steps of a task, such that for at least one step of the plurality of steps of the task, while the operator is navigated to the step of the task: the mixed-reality view is caused to include at least one instruction associated with the step. The mixed-reality view is caused to include a hologram at a real-world location in the real-world environment at which work associated with the step is to be performed. The mixed-reality view is caused to continually include a visual tether from the instruction to the real-world location.

Ergonomic mixed reality step-by-step instructions tethered to 3D holograms in real-world locations

Various methods and systems, for implementing three-dimensional resource integration, are provided. 3D resource integration includes integration of 3D resources into different types of functionality, such as, operating system, file explorer, application and augmented reality functionality. In operation, an indication to perform an operation with a 3D object is received. One or more 3D resource controls, associated with the operation, are accessed. The 3D resource control is a defined set of instructions on how to integrate 3D resources with 3D objects for generating 3D-based graphical interfaces associated with application features and operating system features. An input based on one or more control elements of the one or more 3D resource controls is received. The input includes the one or more control elements that operate to generate a 3D-based graphical interface for the operation. Based on receiving the input, the operation is executed with the 3D object and the 3D-based graphical interface.

Three-dimensional resource integration system

A system recognizes human beings in their natural environment, without special sensing devices attached to the subjects, uniquely identifies them and tracks them in three dimensional space. The resulting representation is presented directly to applications as a multi-point skeletal model delivered in real-time. The device efficiently tracks humans and their natural movements by understanding the natural mechanics and capabilities of the human muscular-skeletal system. The device also uniquely recognizes individuals in order to allow multiple people to interact with the system via natural movements of their limbs and body as well as voice commands/responses.

Device for identifying and tracking multiple humans over time

A capture device may capture a user's motion and a display device may display a model that maps to the user's motion, including gestures that are applicable for control. A user may be unfamiliar with a system that maps the user's motions or not know what gestures are applicable for an executing application. A user may not understand or know how to perform gestures that are applicable for the executing application. Providing visual feedback representing instructional gesture data to the user can teach the user how to properly gesture. The visual feedback may be provided in any number of suitable ways. For example, visual feedback may be provided via ghosted images, player avatars, or skeletal representations. The system can process prerecorded or live content for displaying visual feedback representing instructional gesture data. The feedback can portray the deltas between the user's actual position and the ideal gesture position.

Show body position

It may be desirable to apply corrective data to aspects of captured image or the user-performed gesture for display of a visual representation that corresponds to the corrective data. The captured motion may be any motion in the physical space that is captured by the capture device, such as a camera. Aspects of a skeletal or mesh model of a person, that is generated based on the image data captured by the capture device, may be modified prior to animation. The modification may be made to the model generated from image data that represents a target or a target's motion, including user gestures, in the physical space. For example, certain joints of a skeletal model may be readjusted or realigned. A model of a target may be modified by applying differential correction, magnetism principles, binary snapping, confining virtual movement to defined spaces, or the like.

Determine intended motions

Growing community of inventors

Vancouver, Canada

Darren Alexander Bennett