Microsoft Technology Licensing, LLC

University of Arizona

Dmitry Reshidko

Bernard Charles Kress

Richard Andrew Wall

Dmitriy Churin

Tuomas Vallius

Robert Thomas Held

Ishan Chatterjee

Ian Anh Nguyen

Joseph Daniel Lowney

Parry Byron Johnson

Ashley Saulsbury

David Douglas Bohn

William J Cummings

Tuomas Heikki Sakari Vallius

Curtis Alan Tesdahl

David Rohn

Benjamin Eliot Lundell

R Andrew Wall

Pietari Tuomisto

Sayyed Jaffar Ali Raza

Kevin James Matherson

Congshan Wan

Brian M Wheelwright

Steven John Robbins

Adrian Travis

Michael Edward Samples

Roger P Angel

Richard Allen James

Abhijit Sarkar

Rayna Demaster-Smith

Justin J Hyatt

Heikki Juhana Hyvarinen

Nicholas Mark Cirucci

Nicholas J Didato

Tero Ollikainen

Richard A James

Richard A Wall

Examples are disclosed that relate to calibration of a stereoscopic display system of an HMD via an optical calibration system comprising a waveguide combiner. One example provides an HMD device comprising a first image projector and a second image projector configured to project a stereoscopic image pair, and an optical calibration system. The optical calibration system comprises a first optical path indicative of an alignment of the first image projector, a second optical path indicative of an alignment of the second image projector, a waveguide combiner in which the first and second optical paths combine into a shared optical path, and one or more boresight sensors configured to detect calibration image light traveling along one or more of the first optical or the second optical path.

Calibration of stereoscopic display using waveguide combiner

Eye and hand tracking systems in head-mounted display (HMD) devices are arranged with lensless camera systems using optical masks as encoding elements that apply convolutions to optical images of body parts (e.g., eyes or hands) of HMD device users. The convolved body images are scrambled or coded representations that are captured by a sensor in the system, but are not human-recognizable. A machine learning system such as a neural network is configured to extract body features directly from the coded representation without performance of deconvolutions conventionally utilized to reconstruct the original body images in human-recognizable form. The extracted body features are utilized by the respective eye or hand tracking systems to output relevant tracking data for the user's eyes or hands which may be utilized by the HMD device to support various applications and user experiences. The lensless camera and machine learning system are jointly optimizable on an end-to-end basis.

Eye and hand tracking utilizing lensless camera and machine learning

A projection system includes an illumination light source configured to emit an illumination light beam, a monitor light source configured to emit a monitor light beam, and a projector configured to project both the illumination light beam and the monitor light beam into a projected combined light beam. A first portion of the projected combined light beam is propagated over a first beam path in a first direction, causing an eye of a user to see a display image. A second portion of the projected combined light beam is propagated over a second beam path in a second direction, causing a monitor camera to capture a monitor image. The monitor image is analyzed to determine an orientation or a position of the monitor image. In response to determining that the monitor image is not properly oriented or positioned, an orientation or position of the projector or the illumination image is adjusted.

Monitoring position and orientation of a projector

A display engine adapted for use in a head-mounted display (HMD) device includes a reflective liquid crystal on silicon (LCoS) spatial light modulator (SLM) that is illuminated using a backlight illumination module and a pair of optical prisms providing a total internal reflection (TIR) function. In an illustrative mixed-reality embodiment, the TIR prism pair guides light to the LCoS SLM from the backlight illumination module and projects virtual images reflected from the LCoS SLM, through projection optics, to a diffractive waveguide combiner for viewing by an HMD user.

TIR prisms and use of backlight for LCoS microdisplay illumination

Variable-focus lenses are arranged as a lens pair that work on opposite sides of a see-through optical combiner used in a mixed-reality head-mounted display (HMD) device. An eye-side variable-focus lens is configured as a negative lens over an eyebox of the see-through optical combiner to enable virtual-world objects to be set at a close distance. The negative lens is compensated by its conjugate using a real-world-side variable-focus lens configured as a positive lens to provide an unperturbed see-through experience. For non-presbyopes, the powers of the lenses are perfectly offset. For presbyopes, the lens powers is mismatched at times to provide simultaneous views of both virtual-world and real-world objects on the display in sharp focus. Responsively an eye tracker indicating that the user is engaged in close viewing, optical power is added to the real-world-side lens to push close real-world objects optically farther away into sharp focus for the presbyopic user.

Control of variable-focus lenses in a mixed-reality device for presbyopes

An optical combiner in a display system of a mixed-reality head-mounted display (HMD) device comprises a lens of birefringent material and a ferroelectric liquid crystal (FLC) modulator that are adapted for use with a reflective waveguide to provide multiple different focal planes on which holograms of virtual-world objects (i.e., virtual images) are displayed. The birefringent lens has two orthogonal refractive indices, ordinary and extraordinary, depending on the polarization state of the incident light. Depending on the rotation of the polarization axis by the FLC modulator, the incoming light to the birefringent lens is focused either at a distance corresponding to the ordinary refractive index or the extraordinary refractive index. Virtual image light leaving the birefringent lens is in-coupled to a see-through reflective waveguide which is configured to form an exit pupil for the optical combiner to enable an HMD device user to view the virtual images from the source.

Combined birefringent material and reflective waveguide for multiple focal planes in a mixed-reality head-mounted display device

Examples are disclosed that relate to display devices having a common light path region. One example provides a display device comprising a light source configured to emit illumination light along an illumination path, and a spatial light modulator configured to modulate the illumination light and emit the modulated illumination light as image light along an imaging path, wherein at least a portion of the illumination path and at least a portion of the imaging path extend through a common light path region. The display device further comprises one or more optical elements positioned within the common light path region, at least one optical element being configured to guide the illumination light as the illumination light travels through the common light path region toward the spatial light modulator, and shape the image light as the image light travels through the common light path region.

Display device having common light path region

Variable-focus lenses are arranged as a lens pair that work on opposite sides of a see-through optical combiner used in a mixed-reality head-mounted display (HMD) device. An eye-side variable-focus lens is configured as a negative lens over an eyebox of the see-through optical combiner to enable virtual-world objects to be set at a close distance. The negative lens is compensated by its conjugate using a real-world-side variable-focus lens configured as a positive lens to provide for an unperturbed see-through experience. For non-presbyopes, the powers of the lenses are perfectly offset. For presbyopes, the lens powers may be mismatched at times to provide simultaneous views of both virtual-world and real-world objects on the display in sharp focus. Responsively an eye tracker indicating that the user is engaged in close viewing, optical power is added to the real-world-side lens to push close real-world objects optically farther away and into sharp focus for the presbyopic user.

A near-eye display device comprises right and left display projectors, expansion optics, and inertial measurement units (IMUs), in addition to a plurality of angle-sensitive pixel (ASP) elements and a computer. The right and left expansion optics are configured to receive respective display images from the right and left display projectors and to release expanded forms of the display images. The right IMU is fixedly coupled to the right display projector, and the left IMU is fixedly coupled to the left display projector. Each ASP element is responsive to an angle of light of one of the respective display images as received into the right or left expansion optic. The computer is configured to receive output from the right IMU, the left IMU and the plurality of ASP elements, and render display data for the right and left display projectors based in part on the output.

Autocalibrated near-eye display

In a see-through waveguide-based HMD device configured to display holographic virtual images within a field of view (FOV) of the device user, a single pixel or group of pixels are lit to supply illumination at known locations on the display that is reflected from the user's eyes and captured by one or more sensors in an eye tracker. The eye tracker may apply real-time image analysis to the captured reflected light, called 'glints,' to extract features of the user's eyes to determine where the HMD device user is looking&#x2014;the gaze point&#x2014;and calculate eye movement, location, and orientation. A negative lens functionality utilized in the HMD device to provide a fixed focal depth for the virtual images enables the lit pixels to function as virtual glint sources for the eye tracker sensor that are located at the fixed focal depth and from multiple illumination positions within the user's FOV.

Eye tracker illumination through a waveguide

A method for setting a display white point comprises displaying images with a display including at least a first light source and a second light source. The first light source is configured to emit light having a first color and having a first temperature-dependent luminance change. The second light source is configured to emit light having a second color and having a second temperature-dependent luminance change. An internal display temperature is measured. Based on the internal display temperature being a first temperature, a first target white point is set to prioritize color accuracy. Based on the internal display temperature being a second temperature, greater than the first temperature, a second target white point is set to prioritize luminance output.

Setting white point based on display temperature

An apparatus for generating electricity from solar energy has a large dish reflector with a fly's eye receiver positioned near the focus of the dish reflector, held by a dual axis tracking structure. The fly's eye receiver includes a field lens that concentrates sunlight into an image of the dish reflector, a two-dimensional fly's eye array of contiguous convex lenses extending across the dish image, and a photovoltaic cell behind each convex lens of the fly's eye array. Two imaging stages are provided. First, the dish reflector and the field lens concentrate the sunlight in the form of an image of the dish that is stabilized against pointing errors of the tracking mechanism. Second, the contiguous array of convex lenses divides the sunlight energy of the dish image into portions, one per convex lens, each portion being further concentrated by the respective convex lens onto a corresponding photovoltaic cell.

Solar PV generator with dish reflector and fly's eye receiver

Growing community of inventors

Redmond, WA, United States of America

Dmitry Reshidko