Microsoft Technology Licensing, LLC

Georg Klein

Arthur Charles Tomlin

Drew Edward Steedly

Frederik Schaffalitzky

Ashraf A Michail

Jeffrey Neil Margolis

Andrew Martin Pearson

Calvin Chan

Ethan Eade

Adam G Poulos

Michael Isard

Kathleen Patricia Mulcahy

Nathan Ackerman

Michael Ebstyne

Martin Thomas Shetter

Tony Ambrus

Jeffrey Cole

Ian Douglas McIntyre

Paul Ronald Barham

Kevin Woo

Oliver Michael Christian Williams

Douglas Kevin Service

Tuan Wong

David Nister

Avi Bar-Zeev

John R Lewis

Alex A Kipman

Mark Stefan Grossman

Zsolt Mathe

Navid Poulad

Ning Xu

Michael Grabner

Rayna Demaster-Smith

Ashraf Ayman Michail

Ian Douglas Mclntyre

The techniques disclosed herein detect sensor misalignments in a display device by the use of sensors operating under different modalities. In some configurations, a near-to-eye display device can include a number of sensors that can be used to track movement of the device relative to a surrounding environment. The device can utilize multiple sensors operating under multiple modalities. For each sensor, there is a set of intrinsic and extrinsic properties that are calibrated. The device is also configured to determine refined estimations of the intrinsic and extrinsic properties at runtime. The refined estimations of the intrinsic and extrinsic properties can then be used to derive knowledge on how the device has deformed over time. The device can then use the refined estimations of the intrinsic and extrinsic properties and/or any other resulting data that quantifies any deformations of the device to make adjustments to rendered images at runtime.

Enhanced detection of sensor misalignments by coordinating sensors utilizing multiple sensing modalities

Embodiments are disclosed for methods and systems of distinguishing movements of features in a physical environment. For example, on a head-mounted display device, one embodiment of a method includes obtaining a representation of real-world features in two or more coordinate frames and obtaining motion data from one or more sensors external to the head-mounted display device. The method further includes distinguishing features in one coordinate frame from features in another coordinate frame based upon the motion data.

Determining coordinate frames in a dynamic environment

Examples are disclosed that relate to depth-aware late-stage reprojection. One example provides a computing system configured to receive and store image data, receive a depth map for the image data, processing the depth map to obtain a blurred depth map, and based upon motion data, determine a translation to be made to the image data. Further, for each pixel, the computing system is configured to translate an original ray extending from an original virtual camera location to an original frame buffer location to a reprojected ray extending from a translated camera location to a reprojected frame buffer location, determine a location at which the reprojected ray intersects the blurred depth map, and sample a color of a pixel for display based upon a color corresponding to the location at which the reprojected ray intersects the blurred depth map.

Depth-aware reprojection

Embodiments related to mapping an environment of a machine-vision system are disclosed. For example, one disclosed method includes acquiring image data resolving one or more reference features of an environment and computing a parameter value based on the image data, wherein the parameter value is responsive to physical deformation of the machine-vision system.

3D mapping with flexible camera rig

Methods for generating and displaying images associated with one or more virtual objects within an augmented reality environment at a frame rate that is greater than a rendering frame rate are described. The rendering frame rate may correspond with the minimum time to render images associated with a pose of a head-mounted display device (HMD). In some embodiments, the HMD may determine a predicted pose associated with a future position and orientation of the HMD, generate a pre-rendered image based on the predicted pose, determine an updated pose associated with the HMD subsequent to generating the pre-rendered image, generate an updated image based on the updated pose and the pre-rendered image, and display the updated image on the HMD. The updated image may be generated via a homographic transformation and/or a pixel offset adjustment of the pre-rendered image.

Late stage reprojection

A method and system that enhances a user's experience when using a near eye display device, such as a see-through display device or a head mounted display device is provided. An optimized image for display relative to the a field of view of a user in a scene is created. The user's head and eye position and movement are tracked to determine a focal region for the user. A portion of the optimized image is coupled to the user's focal region in the current position of the eyes, a next position of the head and eyes predicted, and a portion of the optimized image coupled to the user's focal region in the next position.

Optimized focal area for augmented reality displays

An augmented reality device including a plurality of sensors configured to output pose information indicating a pose of the augmented reality device. The augmented reality device further includes a band-agnostic filter and a band-specific filter. The band-specific filter includes an error correction algorithm configured to receive pose information as filtered by the band-agnostic filter and reduce a tracking error of the pose information in a selected frequency band. The augmented reality device further includes a display engine configured to position a virtual object on a see-through display as a function of the pose information as filtered by the band-agnostic filter and the band-specific filter.

Pose tracking an augmented reality device

A system and related methods for near-eye display of an image are provided. In one example, a near-eye display system includes a light source comprising a surface and a plurality of pixels having a pixel pitch of 5 microns or less. An aperture array is located between 2 mm and 5 mm from the surface of the light source. The aperture array comprises non-overlapping apertures that are each centered on a vertex of an equilateral triangle within a grid of equilateral triangles. The center of each aperture is spaced from the center of each adjacent aperture by an aperture spacing of between 1 mm and 9 mm. The aperture array selectively passes the light emitted from the pixels to display the image without a double image condition.

Near eye display

Growing community of inventors

Seattle, WA, United States of America

Georg Klein