Microsoft Technology Licensing, LLC

Dmitriy Churin

Parry Byron Johnson

Dmitry Reshidko

James Peele Terrell, Jr

Michael Scott Sutton

Curtis Alan Tesdahl

David Rohn

Benjamin Eliot Lundell

Rayna DeMaster-Smith

Sayyed Jaffar Ali Raza

Kevin James Matherson

Congshan Wan

Ishan Chatterjee

Rayna Demaster-Smith

Examples are disclosed that relate to calibration of a stereoscopic display system of an HMD via an optical calibration system comprising a waveguide combiner. One example provides an HMD device comprising a first image projector and a second image projector configured to project a stereoscopic image pair, and an optical calibration system. The optical calibration system comprises a first optical path indicative of an alignment of the first image projector, a second optical path indicative of an alignment of the second image projector, a waveguide combiner in which the first and second optical paths combine into a shared optical path, and one or more boresight sensors configured to detect calibration image light traveling along one or more of the first optical or the second optical path.

Calibration of stereoscopic display using waveguide combiner

A head-mounted display system includes a left display assembly configured to provide left-side display light. A first left inertial measurement unit (IMU) is configured to measure an orientation of the left display assembly. A right display assembly is configured to provide right-side display light. A first right IMU is configured to measure an orientation of the right display assembly. A communication interface is configured to receive a left-side orientation of a head-tracking system as measured by a second left IMU, and a right-side orientation of a head-tracking system as measured by a second right IMU. A logic machine is configured to assess an alignment of the head-mounted display system based at least in part on the orientation of the left display assembly, the orientation of the right display assembly, the left-side orientation of the head-tracking system, and the right-side orientation of the head-tracking system.

Head-mounted display alignment assessment

A head-mounted display system includes a left display assembly configured to provide left-side display light and left-side test light. A left waveguide incouples the left-side display light and outcouples the left-side display light for viewing. A left optical sensor is positioned to measure the left-side test light. A left inertial measurement unit (IMU) is configured to measure an orientation of the left display assembly. A right display assembly is configured to provide right-side display light and right-side test light. A right waveguide incouples the right-side display light and outcouples the right-side display light for viewing. A right optical sensor is positioned to measure the right-side test light. A right IMU is configured to measure an orientation of the right display assembly. A logic machine is configured to assess a stereo alignment for the left- and right-side display light.

Stereo alignment assessment for head-mounted display

Eye and hand tracking systems in head-mounted display (HMD) devices are arranged with lensless camera systems using optical masks as encoding elements that apply convolutions to optical images of body parts (e.g., eyes or hands) of HMD device users. The convolved body images are scrambled or coded representations that are captured by a sensor in the system, but are not human-recognizable. A machine learning system such as a neural network is configured to extract body features directly from the coded representation without performance of deconvolutions conventionally utilized to reconstruct the original body images in human-recognizable form. The extracted body features are utilized by the respective eye or hand tracking systems to output relevant tracking data for the user's eyes or hands which may be utilized by the HMD device to support various applications and user experiences. The lensless camera and machine learning system are jointly optimizable on an end-to-end basis.

Eye and hand tracking utilizing lensless camera and machine learning

A projection system includes an illumination light source configured to emit an illumination light beam, a monitor light source configured to emit a monitor light beam, and a projector configured to project both the illumination light beam and the monitor light beam into a projected combined light beam. A first portion of the projected combined light beam is propagated over a first beam path in a first direction, causing an eye of a user to see a display image. A second portion of the projected combined light beam is propagated over a second beam path in a second direction, causing a monitor camera to capture a monitor image. The monitor image is analyzed to determine an orientation or a position of the monitor image. In response to determining that the monitor image is not properly oriented or positioned, an orientation or position of the projector or the illumination image is adjusted.

Monitoring position and orientation of a projector

The techniques disclosed herein may be utilized to detect, measure, and/or compensate for misalignments of a display that may occur after manufacturing. A Talbot sensor is described that includes a diffraction device and an image sensor. Captured images from the image sensor include pixel data values that include bright and dark spots that represent a diffraction pattern associated with the Talbot sensor. A demodulator multiplies the pixel data values with sine and cosine reference images to generate averaged in-phase and quadrature values, which can be used to determine a phase angle for incident light on the Talbot sensor. Phase angle changes over time indicate changes in the alignment of the display, which may be corrected by display parameter manipulation. The resulting devices, systems and methods provide for portable solutions, with reduced cost of manufacturing, reduced part costs, and reduced complexity.

Growing community of inventors

Redmond, WA, United States of America

Dmitriy Churin