Microsoft Technology Licensing, LLC

Jeff Cole

Daniel Joseph McCulloch

Adam G Poulos

Cameron Graeme Brown

David Douglas Bohn

Stephen G Latta

Rod G Fleck

Kathleen Patricia Mulcahy

Jeffrey Alan Kohler

Aaron Burns

Brian C McDowell

Julia Louise Meinershagen

Todd Omotani

Sebastian Sylvan

Adam Benjamin Smith-Kipnis

Jonathan Plumb

Abby Lin Lee

Hendrik Mark Langerak

Michael Gordon Hale

Felicia Stephanie Williams

Tim Psiaki

Alexandre David

Daniel Mcculloch

Michael O Hale

Jonathan William Plumb

Felicia Williams

Embodiments are disclosed that relate to operating a user interface on an augmented reality computing device comprising a see-through display system. For example, one disclosed embodiment includes receiving a user input selecting an object in a field of view of the see-through display system, determining a first group of commands currently operable based on one or more of an identification of the selected object and a state of the object, and presenting the first group of commands to a user. The method may further include receiving a command from the first group of commands, changing the state of the selected object from a first state to a second state in response to the command, determining a second group of commands based on the second state, where the second group of commands is different than the first group of commands, and presenting the second group of commands to the user.

Context-aware augmented reality object commands

A user input for a near-eye, see-through display device is disclosed. Hands-free user input in an augmented reality environment is provided for. A user can provide input by moving the orientation of their head. For example, the user could rotate their head. In one aspect, a user can provide input by moving their eye gaze along a direction. In one aspect, when the user directs their attention at a user interface symbol, a handle extends away from the user interface symbol. The handle may serve as a type of selection device such that if the user directs their attention along the handle, away from the user interface symbol, a selection can be made. 'As one example, the selection causes a spoke menu to appear which the user can select by rotating their head such that the system determines the user is looking along the spoke away from a central hub.

User focus controlled directional user input

Embodiments that relate to converting audio inputs from an environment into text are disclosed. For example, in one disclosed embodiment a speech conversion program receives audio inputs from a microphone array of a head-mounted display device. Image data is captured from the environment, and one or more possible faces are detected from image data. Eye-tracking data is used to determine a target face on which a user is focused. A beamforming technique is applied to at least a portion of the audio inputs to identify target audio inputs that are associated with the target face. The target audio inputs are converted into text that is displayed via a transparent display of the head-mounted display device.

Speech to text conversion

A blocking image generating system and related methods include a head-mounted display device having an opacity layer. A method may include receiving a virtual image to be presented by display optics in the head-mounted display device. Lighting information and an eye-position parameter may be received from an optical sensor system in the head-mounted display device. A blocking image may be generated in the opacity layer of the head-mounted display device based on the lighting information and the virtual image. The location of the blocking image in the opacity layer may be adjusted based on the eye-position parameter.

Display with blocking image generation

Growing community of inventors

Seattle, WA, United States of America

Jeff Cole