Microsoft Technology Licensing, LLC

Stephen G Latta

Kevin Geisner

Relja Markovic

Darren Bennett

Daniel Joseph McCulloch

Ryan Lucas Hastings

Brian James Mount

Jason Bradley Scott

Jonathan Steed

Arthur Charles Tomlin

Ben J Sugden

Gregory Nelson Snook

Kathryn Stone Perez

Benjamin Isaac Vaught

Alex Aben-Athar Kipman

Aaron Daniel Krauss

Mike Scavezze

Michael Scavezze

Shawn Crispin Wright

John Clavin

Brian Murphy

Ian Douglas McIntyre

Adam G Poulos

Kudo Tsunoda

Wei Zhang

Tony Ambrus

Alex A Kipman

Jeffrey Neil Margolis

Cameron Graeme Brown

Robert L Crocco, Jr

Brian E Keane

Christopher Michael Novak

Peter Tobias Kinnebrew

Christopher Harley Willoughby

Joel Deaguero

Rod G Fleck

Mark James Finocchio

Nicholas Ferianc Kamuda

Jeffrey Alan Kohler

Sheridan Martin Small

Steven Nabil Bathiche

Laura K Massey

Kyungsuk David Lee

Johnny Hsienchow Lee

Craig R Maitlen

Mathew Julian Lamb

Vivek Pradeep

Nicholas Fajt

Darren Alexander Bennett

Daniel Deptford

John Bevis

Thomas Salter

Jamie Bryant Kirschenbaum

David Douglas Bohn

Kenneth Paul Hinckley

Hrvoje Benko

Andrew John Fuller

Alice Jane Bernheim Brush

Alexandru Octavian Balan

Drew Edward Steedly

Frederik Schaffalitzky

Janet Lynn Schneider

Anatolie Gavriliuc

Christian Michael Sadak

Jeffrey Jesus Evertt

Dan Osborn

Kenneth Liam Kiemele

Jeffrey Sipko

Donna Katherine Long

Jeffrey Cole

Anton O Andrews

Christopher E Miles

Kendall Clark York

Adam Jethro Langridge

Bryant Daniel Hawthorne

Amos Dylan Vance

Rudy Jacobus Poot

David M Hill

Forest Woodcroft Gouin

Megan Saunders

Desney S Tan

Jeffrey C Fong

Ethan Eade

Derek L Knee

Ashraf A Michail

Aaron N Woo

R Stephen Polzin

Michael Ebstyne

Robert Thomas Held

Martin Thomas Shetter

Brian C McDowell

Scott Robert Ramsby

Julia Louise Meinershagen

Britta Silke Hummel

David Seymour

Robert Pengelly

Jedd Perry

Paul Albert Lalonde

Tarek Hefny

Andrew Wyman MacDonald

A system for automatically displaying virtual objects within a mixed reality environment is described. In some embodiments, a see-through head-mounted display device (HMD) identifies a real object (e.g., a person or book) within a field of view of the HMD, detects one or more interactions associated with real object, and automatically displays virtual objects associated with the real object if the one or more interactions involve touching or satisfy one or more social rules stored in a social rules database. The one or more social rules may be used to infer a particular social relationship by considering the distance to another person, the type of environment (e.g., at home or work), and particular physical interactions (e.g., handshakes or hugs). The virtual objects displayed on the HMD may depend on the particular social relationship inferred (e.g., a friend or acquaintance).

Touch and social cues as inputs into a computer

Embodiments related to efficiently constructing an augmented reality environment with global illumination effects are disclosed. For example, one disclosed embodiment provides a method of displaying an augmented reality image via a display device. The method includes receiving image data, the image data capturing an image of a local environment of the display device, and identifying a physical feature of the local environment via the image data. The method further includes constructing an augmented reality image of a virtual structure for display over the physical feature in spatial registration with the physical feature from a viewpoint of a user, the augmented reality image comprising a plurality of modular virtual structure segments arranged in adjacent locations to form the virtual structure feature, each modular virtual structure segment comprising a pre-computed global illumination effect, and outputting the augmented reality image to the display device.

Constructing augmented reality environment with pre-computed lighting

Systems, methods and computer readable media are disclosed for gesture input beyond skeletal. A user's movement or body position is captured by a capture device of a system. Further, non-user-position data is received by the system, such as controller input by the user, an item that the user is wearing, a prop under the control of the user, or a second user's movement or body position. The system incorporates both the user-position data and the non-user-position data to determine one or more inputs the user made to the system.

Combining gestures beyond skeletal

Examples are disclosed that relate to devices and methods for sharing geo-located information between different devices. In one example, a method comprises receiving the geo-located information from a first user device having a first data type compatible with a first output component of the device, receiving first sensor data from the first device, determining a location of the geo-located information within a coordinate system in a physical environment, determining that a second user device is located in the physical environment, determining that the second device does not comprise an output component that is compatible with the first data type, transforming the geo-located information into a second data type compatible with a second output component of the second device, determining that the second device is proximate to the location of the geo-located information, and sending the geo-located information to the second device for output by the second output component.

Sharing geo-located information

Embodiments that relate to interacting with a physical object in a mixed reality environment via a head-mounted display are disclosed. In one embodiment a mixed reality interaction program identifies an object based on an image from captured by the display. An interaction context for the object is determined based on an aspect of the mixed reality environment. A profile for the physical object is queried to determine interaction modes for the object. A selected interaction mode is programmatically selected based on the interaction context. A user input directed at the object is received via the display and interpreted to correspond to a virtual action based on the selected interaction mode. The virtual action is executed with respect to a virtual object associated with the physical object to modify an appearance of the virtual object. The modified virtual object is then displayed via the display.

Mixed reality interactions

A head-mounted display includes a see-through display and a virtual reality engine. The see-through display is configured to visually augment an appearance of a physical space to a user viewing the physical space through the see-through display. The virtual reality engine is configured to cause the see-through display to visually present a virtual monitor that appears to be integrated with the physical space to a user viewing the physical space through the see-through display.

Augmented reality virtual monitor

The technology provides contextual personal information by a mixed reality display device system being worn by a user. A user inputs person selection criteria, and the display system sends a request for data identifying at least one person in a location of the user who satisfy the person selection criteria to a cloud based application with access to user profile data for multiple users. Upon receiving data identifying the at least one person, the display system outputs data identifying the person if he or she is within the field of view. An identifier and a position indicator of the person in the location is output if not. Directional sensors on the display device may also be used for determining a position of the person. Cloud based executing software can identify and track the positions of people based on image and non-image data from display devices in the location.

Providing location occupancy analysis via a mixed reality device

The technology causes disappearance of a real object in a field of view of a see-through, mixed reality display device system based on user disappearance criteria. Image data is tracked to the real object in the field of view of the see-through display for implementing an alteration technique on the real object causing its disappearance from the display. A real object may satisfy user disappearance criteria by being associated with subject matter that the user does not wish to see or by not satisfying relevance criteria for a current subject matter of interest to the user. In some embodiments, based on a 3D model of a location of the display device system, an alteration technique may be selected for a real object based on a visibility level associated with the position within the location. Image data for alteration may be prefetched based on a location of the display device system.

User controlled real object disappearance in a mixed reality display

A system for generating a virtual gaming environment based on features identified within a real-world environment, and adapting the virtual gaming environment over time as the features identified within the real-world environment change is described. Utilizing the technology described, a person wearing a head-mounted display device (HMD) may walk around a real-world environment and play a virtual game that is adapted to that real-world environment. For example, the HMD may identify environmental features within a real-world environment such as five grassy areas and two cars, and then spawn virtual monsters based on the location and type of the environmental features identified. The location and type of the environmental features identified may vary depending on the particular real-world environment in which the HMD exists and therefore each virtual game may look different depending on the particular real-world environment.

Augmented reality spaces with adaptive rules

Methods for recognizing gestures using adaptive multi-sensor gesture recognition are described. In some embodiments, a gesture recognition system receives a plurality of sensor inputs from a plurality of sensor devices and a plurality of confidence thresholds associated with the plurality of sensor inputs. A confidence threshold specifies a minimum confidence value for which it is deemed that a particular gesture has occurred. Upon detection of a compensating event, such as excessive motion involving one of the plurality of sensor devices, the gesture recognition system may modify the plurality of confidence thresholds based on the compensating event. Subsequently, the gesture recognition system generates a multi-sensor confidence value based on whether at least a subset of the plurality of confidence thresholds has been satisfied. The gesture recognition system may also modify the plurality of confidence thresholds based on the plugging and unplugging of sensor inputs from the gesture recognition system.

Multiple sensor gesture recognition

Embodiments are disclosed that relate to authenticating a user of a display device. For example, one disclosed embodiment includes displaying one or more virtual images on the display device, wherein the one or more virtual images include a set of augmented reality features. The method further includes identifying one or more movements of the user via data received from a sensor of the display device, and comparing the identified movements of the user to a predefined set of authentication information for the user that links user authentication to a predefined order of the augmented reality features. If the identified movements indicate that the user selected the augmented reality features in the predefined order, then the user is authenticated, and if the identified movements indicate that the user did not select the augmented reality features in the predefined order, then the user is not authenticated.

Multi-input user authentication on display device

Embodiments are disclosed herein that relate to the automatic tracking of objects. For example, one disclosed embodiment provides a method of operating a mobile computing device having an image sensor. The method includes acquiring image data, identifying an inanimate moveable object in the image data, determining whether the inanimate moveable object is a tracked object, if the inanimate moveable object is a tracked object, then storing information regarding a state of the inanimate moveable object, detecting a trigger to provide a notification of the state of the inanimate moveable object, and providing an output of the notification of the state of the inanimate moveable object.

Object tracking

An example wearable display system includes a controller, a left display to display a left-eye augmented reality image with a left-eye display size at left-eye display coordinates, and a right display to display a right-eye augmented reality image with a right-eye display size at right-eye display coordinates, the left-eye and right-eye augmented reality images collectively forming an augmented reality object perceivable at an apparent real world depth by a wearer of the display system. The controller sets the left-eye display coordinates relative to the right-eye display coordinates as a function of the apparent real world depth of the augmented reality object. The function maintains an aspect of the left-eye and right-eye display sizes throughout a non-scaling range of apparent real world depths of the augmented reality object, and the function scales the left-eye and right-eye display sizes with changing apparent real world depth outside the non-scaling range.

Fixed size augmented reality objects

Growing community of inventors

Bothell, WA, United States of America

Stephen G Latta