Microsoft Technology Licensing, LLC

Microsoft Corporation

John Clavin

Kathryn Stone Perez

Stephen G Latta

Kevin Geisner

Ben J Sugden

Jason Bradley Scott

Daniel Joseph McCulloch

Arthur Charles Tomlin

Darren Bennett

Ryan Lucas Hastings

Alex Aben-Athar Kipman

Benjamin Isaac Vaught

Wei Zhang

Ben Vaught

Alex A Kipman

Megan Lesley Tedesco

Kudo Tsunoda

Steven Nabil Bathiche

John Allen Tardif

Daniel John Wigdor

Brian James Mount

Adam G Poulos

Todd E Holmdahl

Otto G Berkes

Ian C LeGrow

Joseph Reginald Scott Molnar

Aaron E Kornblum

Eyal Ofek

Scott Charles McEldowney

Robert G Walker

Giora Yahav

Relja Markovic

Thomas Charles Oliver

David Cohen

Miller Thomas Abel

Jonathan Steed

Kenneth Alan Lobb

Doug Burger

Kamran Rajabi Zargahi

Gregory Nelson Snook

Russell E Glaser

Viswanath Vadlamani

Joseph Futty

Peter Glenn Sarrett

Scott Henson

Steven Michael Beeman

Joseph Molnar

Drew Angeloff

Eric P Gilmore

Gaelle Vialle

Robert Sheldon Walker

A system to present the user a 3-D virtual environment as well as non-visual sensory feedback for interactions that user makes with virtual objects in that environment is disclosed. In an exemplary embodiment, a system comprising a depth camera that captures user position and movement, a three-dimensional (3-D) display device that presents the user a virtual environment in 3-D and a haptic feedback device provides haptic feedback to the user as he interacts with a virtual object in the virtual environment. As the user moves through his physical space, he is captured by the depth camera. Data from that depth camera is parsed to correlate a user position with a position in the virtual environment. Where the user position or movement causes the user to touch the virtual object, that is determined, and corresponding haptic feedback is provided to the user.

Systems and methods for immersive interaction with virtual objects

Disclosed herein are techniques for scaling and translating gestures such that the applicable gestures for control may vary depending on the user's distance from a gesture-based system. The techniques for scaling and translation may take the varying distances from which a user interacts with components of the gesture-based system, such as a computing environment or capture device, into consideration with respect to defining and/or recognizing gestures. In an example embodiment, the physical space is divided into virtual zones of interaction, and the system may scale or translate a gesture based on the zones. A set of gesture data may be associated with each virtual zone such that gestures appropriate for controlling aspects of the gesture-based system may vary throughout the physical space.

Distance scalable no touch computing

A head-mounted display (HMD) provides an augmented view of advertisements to an HMD wearer. In some embodiments, when an advertisement is within an HMD wearer's field of view, the HMD may augment the HMD wearer's view of the advertisement to provide additional information and/or to personalize the advertisement to the HMD wearer. In other embodiments, when an advertisement is within an HMD wearer's field of view, the HMD may augment the HMD wearer's view of the advertisement to remove the advertisement from the HMD wearer's view or to replace the content of the advertisement with non-advertising content.

Augmented view of advertisements

Embodiments for interacting with an executable virtual object associated with a real object are disclosed. In one example, a method for interacting with an executable virtual object associated with a real object includes receiving sensor input from one or more sensors attached to the portable see-through display device, and obtaining information regarding a location of the user based on the sensor input. The method also includes, if the location includes a real object comprising an associated executable virtual object, then determining an intent of the user to interact with the executable virtual object, and if the intent to interact is determined, then interacting with the executable object.

Executable virtual objects associated with real objects

A see-through, near-eye, mixed reality display apparatus for providing translations of real world data for a user. A wearer's location and orientation with the apparatus is determined and input data for translation is selected using sensors of the apparatus. Input data can be audio or visual in nature, and selected by reference to the gaze of a wearer. The input data is translated for the user relative to user profile information bearing on accuracy of a translation and determining from the input data whether a linguistic translation, knowledge addition translation or context translation is useful.

Intelligent translations in personal see through display

A game can be created, shared and played using a personal audio/visual apparatus such as a head-mounted display device (HMDD). Rules of the game, and a configuration of the game space, can be standard or custom. Boundary points of the game can be defined by a gaze direction of the HMDD, by the user's location, by a model of a physical game space such as an instrumented court or by a template. Players can be identified and notified of the availability of a game using a server push technology. For example, a user in a particular location may be notified of the availability of a game at that location. A server manages the game, including storing the rules, boundaries and a game state. The game state can identify players and their scores. Real world objects can be imaged and provided as virtual objects in the game space.

Sharing games using personal audio/visual apparatus

Systems, methods and computer readable media are disclosed for gesture shortcuts. A user's movement or body position is captured by a capture device of a system, and is used as input to control the system. For a system-recognized gesture, there may be a full version of the gesture and a shortcut of the gesture. Where the system recognizes that either the full version of the gesture or the shortcut of the gesture has been performed, it sends an indication that the system-recognized gesture was observed to a corresponding application. Where the shortcut comprises a subset of the full version of the gesture, and both the shortcut and the full version of the gesture are recognized as the user performs the full version of the gesture, the system recognizes that only a single performance of the gesture has occurred, and indicates to the application as such.

Gesture shortcuts

An electronic game system and a method of its operation are provided for virtual space mapping of a variable activity region in physical space. A calibration input may be received from a positioning device of a game controller that indicates waypoints that define an activity region in physical space. A scale factor may be identified between the activity region and an interactive game region in virtual space based on the calibration input. Positioning information may be received from the positioning device that indicates a position of the positioning device within the activity region. The position of the positioning device within the activity region may be mapped to a corresponding virtual position within the interactive game region based on the scale factor.

Virtual space mapping of a variable activity region

Technology is described for providing a personalized sport performance experience with three dimensional (3D) virtual data displayed by a near-eye, augmented reality display of a personal audiovisual (A/V) apparatus. A physical movement recommendation is determined for the user performing a sport based on skills data for the user for the sport, physical characteristics of the user, and 3D space positions for at least one or more sport objects. 3D virtual data depicting one or more visual guides for assisting the user in performing the physical movement recommendation may be displayed from a user perspective associated with a display field of view of the near-eye AR display. An avatar may also be displayed by the near-eye AR display performing a sport. The avatar may perform the sport interactively with the user or be displayed performing a prior performance of an individual represented by the avatar.

Enhancing a sport using an augmented reality display

A system for generating an augmented reality environment in association with one or more attractions or exhibits is described. In some cases, a see-through head-mounted display device (HMD) may acquire one or more virtual objects from a supplemental information provider associated with a particular attraction. The one or more virtual objects may be based on whether an end user of the HMD is waiting in line for the particular attraction or is on (or in) the particular attraction. The supplemental information provider may vary the one or more virtual objects based on the end user's previous experiences with the particular attraction. The HMD may adapt the one or more virtual objects based on physiological feedback from the end user (e.g., if a child is scared). The supplemental information provider may also provide and automatically update a task list associated with the particular attraction.

Personal audio/visual system for providing an adaptable augmented reality environment

Technology is described for representing a physical location at a previous time period with three dimensional (3D) virtual data displayed by a near-eye, augmented reality display of a personal audiovisual (A/V) apparatus. The personal A/V apparatus is identified as being within the physical location, and one or more objects in a display field of view of the near-eye, augmented reality display are automatically identified based on a three dimensional mapping of objects in the physical location. User input, which may be natural user interface (NUI) input, indicates a previous time period, and one or more 3D virtual objects associated with the previous time period are displayed from a user perspective associated with the display field of view. An object may be erased from the display field of view, and a camera effect may be applied when changing between display fields of view.

Representing a location at a previous time period using an augmented reality display

A system and method is disclosed for sensing, storing and using personal trait profile data. Once sensed and stored, this personal trait profile data may be used for a variety of purposes. In one example, a user's personal trait profile data may be accessed and downloaded to different computing systems with which a user may interact so that the different systems may be instantly tuned to the user's personal traits and manner of interaction. In a further example, a user's personal trait profile data may also be used for authentication purposes.

Cloud-based personal trait profile data

In a gesture-based system, gestures may control aspects of a computing environment or application, where the gestures may be derived from a user's position or movement in a physical space. A gesture-based system may have a plurality of modes, each mode a hardware configuration, a software configuration, or a combination thereof. Techniques for transitioning a user's control, via the user's gestures, between different modes enables a system to coordinate controls between multiple modes. For example, while a first mode is active, the user's gestures may control aspects of the first mode. The system may transition the user's control from a control of the first mode to a control of a second mode. The transition may be between hardware, software, or a combination thereof. In another embodiment, reserved gestures that correspond to a first mode that may be executed whether or not a second mode is present.

Application gesture interpretation

Growing community of inventors

Seattle, WA, United States of America

John Clavin