Microsoft Technology Licensing, LLC

Adam G Poulos

Daniel Joseph McCulloch

Arthur Charles Tomlin

Brian James Mount

Cameron Graeme Brown

Stephen G Latta

Anthony James Ambrus

Nicholas Fajt

Victor Tan

Sebastian Lange

Jason Bradley Scott

Jeffrey Alan Kohler

Michael Scavezze

Dan B Kroymann

Wei Zhang

Michael Robert Thomas

John Bevis

Tony Ambrus

Jeffrey Cole

Ian Douglas McIntyre

Gregory Alt

Evan Michael Keibler

Lewey Geselowitz

Marcus Philip Ghaly

Matthew Kaplan

Christopher Douglas Edmonds

Kevin Geisner

Kathryn Stone Perez

Christian Klein

Darren Bennett

Ryan Lucas Hastings

Kathleen Patricia Mulcahy

Drew Edward Steedly

Jonathan Steed

Frederik Schaffalitzky

Aaron Daniel Krauss

John Clavin

Ali M Vassigh

Jonathan Gustav Paulovich

Kim Pascal Pimmel

Constantin Dulu

Justin Avram Clark

Georg Klein

James Dack

Eric Scott Rehmeyer

Jeff Cole

Michael Jason Gourlay

Rotem Bennet

Jeremy Lee

Jon Paulovich

Edward Daniel Parker

Jea Gon Park

Jeffrey C Fong

Nicolas A Denhez

Ben J Sugden

Jeffrey Neil Margolis

Alex A Kipman

Shawn Crispin Wright

Alexandru Octavian Balan

Aaron N Woo

Nicholas Ferianc Kamuda

Peter Tobias Kinnebrew

Mathew Julian Lamb

Dinarte R Morais

Anatolie Gavriliuc

Benjamin Hunter Boesel

Lorenz Henric Jentz

Kudo Tsunoda

Dan Osborn

Michael Edward Samples

Stephane Plante

Scott Robert Ramsby

Lev Cherkashin

Robert Pengelly

Michael Grabner

Forest Woodcroft Gouin

Omer Bilal Orhan

Andrew Austin Jackson

Megan Saunders

Jordan Dale Andersen

Andrew Wyman MacDonald

Brian Kramp

Brendan Reville

Roger Sebastian Sylvan

Jeremy Richard Hill

Cheyne Rory Quin Mathey-Owens

Abby Lin Lee

Andrew Lawrence Mattingly

Varun Ramesh Mani

Roger Sebastian-Kevin Sylvan

Arnulfo Zepeda Navratil

Arjun Dayal

Muhammad Jabir Kapasi

Brent Charles Allen

Christopher Obeso

Johanna Dy Lynn

Jesse Rapczak

Udiyan I Padmanahan

Olivia M Janik

Johnathan Robert Bevis

Technology is described for (3D) space carving of a user environment based on movement through the user environment of one or more users wearing a near-eye display (NED) system. One or more sensors of the NED system provide sensor data from which a distance and direction of movement can be determined. Spatial dimensions for a navigable path can be represented based on user height data and user width data of the one or more users who have traversed the path. Space carving data identifying carved out space can be stored in a 3D space carving model of the user environment. The navigable paths can also be related to position data in another kind of 3D mapping like a 3D surface reconstruction mesh model of the user environment generated from depth images.

Space carving based on human physical data

Embodiments are disclosed that relate to operating a user interface on an augmented reality computing device comprising a see-through display system. For example, one disclosed embodiment includes receiving a user input selecting an object in a field of view of the see-through display system, determining a first group of commands currently operable based on one or more of an identification of the selected object and a state of the object, and presenting the first group of commands to a user. The method may further include receiving a command from the first group of commands, changing the state of the selected object from a first state to a second state in response to the command, determining a second group of commands based on the second state, where the second group of commands is different than the first group of commands, and presenting the second group of commands to the user.

Context-aware augmented reality object commands

In embodiments of a camera-based input device, the input device includes an inertial measurement unit that collects motion data associated with velocity and acceleration of the input device in an environment, such as in three-dimensional (3D) space. The input device also includes at least two visual light cameras that capture images of the environment. A positioning application is implemented to receive the motion data from the inertial measurement unit, and receive the images of the environment from the at least two visual light cameras. The positioning application can then determine positions of the input device based on the motion data and the images correlated with a map of the environment, and track a motion of the input device in the environment based on the determined positions of the input device.

Position-determining input device

Embodiments are disclosed herein that relate to tuning gesture recognition characteristics for a device configured to receive gesture-based user inputs. For example, one disclosed embodiment provides a head-mounted display device including a plurality of sensors, a display configured to present a user interface, a logic machine, and a storage machine that holds instructions executable by the logic machine to detect a gesture based upon information received from a first sensor of the plurality of sensors, perform an action in response to detecting the gesture, and determine whether the gesture matches an intended gesture input. The instructions are further executable to update a gesture parameter that defines the intended gesture input if it is determined that the gesture detected does not match the intended gesture input.

Gesture parameter tuning

A head-mounted display includes a see-through display and a virtual reality engine. The see-through display is configured to visually augment an appearance of a physical space to a user viewing the physical space through the see-through display. The virtual reality engine is configured to cause the see-through display to visually present a virtual monitor that appears to be integrated with the physical space to a user viewing the physical space through the see-through display.

Augmented reality virtual monitor

Technology is described for (3D) space carving of a user environment based on movement through the user environment of one or more users wearing a near-eye display (NED) system. One or more sensors on the near-eye display (NED) system provide sensor data from which a distance and direction of movement can be determined. Spatial dimensions for a navigable path can be represented based on user height data and user width data of the one or more users who have traversed the path. Space carving data identifying carved out space can be stored in a 3D space carving model of the user environment. The navigable paths can also be related to position data in another kind of 3D mapping like a 3D surface reconstruction mesh model of the user environment generated from depth images.

A mixed reality system may comprise a head-mounted display (HMD) device with a location sensor and a base station, mounted a predetermined offset from the location sensor, that emits an electromagnetic field (EMF). An EMF sensor affixed to an object may sense the EMF, forming a magnetic tracking system. The HMD device may determine a relative location of the EMF sensor therefrom and determine a location of the EMF sensor in space based on the relative location, the predetermined offset, and the location of the location sensor. An optical tracking system comprising a marker an optical sensor configured to capture optical data may be included to augment the magnetic tracking system based on the optical data and a location of the optical sensor or marker. The HMD device may display augmented reality images and overlay a hologram corresponding to the location of the EMF sensor over time.

Optically augmenting electromagnetic tracking in mixed reality

A system and method are disclosed for building virtual content from within a virtual environment using virtual tools to build and modify the virtual content.

Building holographic content using holographic tools

Embodiments are disclosed for methods and systems of distinguishing movements of features in a physical environment. For example, on a head-mounted display device, one embodiment of a method includes obtaining a representation of real-world features in two or more coordinate frames and obtaining motion data from one or more sensors external to the head-mounted display device. The method further includes distinguishing features in one coordinate frame from features in another coordinate frame based upon the motion data.

Determining coordinate frames in a dynamic environment

Embodiments that relate to displaying holographic keyboard and hand images in a holographic environment are provided. In one embodiment depth information of an actual position of a user's hand is received from a capture device. The user's hand is spaced by an initial actual distance from the capture device, and a holographic keyboard image is displayed spatially separated by a virtual distance from a holographic hand image. The user's hand is determined to move to an updated actual distance from the capture device. In response, the holographic keyboard image is maintained spatially separated by substantially the virtual distance from the holographic hand image.

Holographic keyboard display

Methods and systems for displaying a computer generated image corresponding to the pose of a real-world object in a mixed reality system. The system may include of a head-mounted display (HMD) device, a magnetic track system and an optical system. Pose data detected by the two tracking systems can be synchronized by a timestamp that is embedded in an electromagnetic field transmitted by the magnetic tracking system. A processor may also be configured to calculate a future pose of the real world object based on a time offset based on the time needed by the HMD to calculate, buffer and generate display output and on data from the two tracking systems, such that the relative location of the computer generated image (CGI) corresponds with the actual location of the real-world object relative to the real world environment at the time the CGI actually appears in the display.

Generating and displaying a computer generated image on a future pose of a real world object

A head-mounted display device may display a holographic element with a portable control device. Image data of a physical environment including the control device may be received and used to generate a three dimensional model of at least a portion of the environment. Using position information of the control device, a holographic element is displayed with the control device. Using the position information, it is determined that the control device is within a predetermined proximity of either a holographic object or a physical object. Based on determining that the control device is within the predetermined proximity, the displayed holographic element is modified.

Control device with holographic element

Motion and/or rotation of an input mechanism can be tracked and/or analyzed to determine limits on a user's range of motion and/or a user's range of rotation in three-dimensional space. The user's range of motion and/or the user's range of rotation in three-dimensional space may be limited by a personal restriction for the user (e.g., a broken arm). The user's range of motion and/or the user's range of rotation in three-dimensional space may additionally or alternatively be limited by an environmental restriction (e.g., a physical object in a room). Accordingly, the techniques described herein can take steps to accommodate the personal restriction and/or the environmental restriction thereby optimizing user interactions involving the input mechanism and a virtual object.

Interaction with virtual objects based on determined restrictions

Disclosed are techniques for performing undo operations on holographic objects in an immersive 3D visual environment. A display system allows the user to undo a given user operation performed on a particular selected holographic object without affecting any other holographic objects, based on a user's gaze and/or other user input. The technique can be implemented in conjunction with a scrollable visual 'timeline' in which multiple past states of the display environment are displayed to the user and are selectable by the user as the target state of the revert operation. Also disclosed is a technique for partially undoing a single continuous user action in a holographic display system.

Holographic display system with undo functionality

Embodiments are disclosed that relate to operating a user interface on an augmented reality computing device comprising a display system. For example, one disclosed embodiment includes displaying a virtual object via the display system as free-floating, detecting a trigger to display the object as attached to a surface, and, in response to the trigger, displaying the virtual object as attached to the surface via the display system. The method may further include detecting a trigger to detach the virtual object from the surface and, in response to the trigger to detach the virtual object from the surface, detaching the virtual object from the surface and displaying the virtual object as free-floating.

Interactions of virtual objects with surfaces

Growing community of inventors

Redmond, WA, United States of America

Adam G Poulos