Microsoft Technology Licensing, LLC

Rayna Demaster-Smith

Navid Poulad

Christopher Maurice Mei

Christopher Charles Aholt

Paulo Ricardo Dos Santos Mendonca

Jeffrey Neil Margolis

Dmitry Reshidko

Ishan Chatterjee

Michael Edward Samples

Georg Klein

Dmitriy Churin

James Peele Terrell, Jr

Michael Scott Sutton

The disclosed techniques provide enhanced eye tracking systems utilizing joint estimation of biological parameters and hardware parameters. A system uses joint estimation of biological parameters, e.g., direction and position of an eye, with concurrent estimation of hardware parameters, e.g., camera position or camera direction, to self-calibrate and provide eye tracking estimations to accommodate for deformations and other changes of a device. Sensor data is used to select hardware parameters of a camera for use in the joint estimation with the biological parameters, where the hardware parameters are estimated based on glint and pupil position of a user. The disclosed techniques include a method to model changes of a device, as well as detect and compensate for them while the eye-tracking device is in normal use, without requiring a factory-calibration procedure to be repeated.

Enhanced eye tracking systems utilizing joint biological and hardware estimations

The techniques disclosed herein may be utilized to detect, measure, and/or compensate for misalignments of a display that may occur after manufacturing. A Talbot sensor is described that includes a diffraction device and an image sensor. Captured images from the image sensor include pixel data values that include bright and dark spots that represent a diffraction pattern associated with the Talbot sensor. A demodulator multiplies the pixel data values with sine and cosine reference images to generate averaged in-phase and quadrature values, which can be used to determine a phase angle for incident light on the Talbot sensor. Phase angle changes over time indicate changes in the alignment of the display, which may be corrected by display parameter manipulation. The resulting devices, systems and methods provide for portable solutions, with reduced cost of manufacturing, reduced part costs, and reduced complexity.

Angle sensing using the Talbot effect, an imaging sensor, and phase/quadrature demodulation

A near-eye display device comprises right and left display projectors, expansion optics, and inertial measurement units (IMUs), in addition to a plurality of angle-sensitive pixel (ASP) elements and a computer. The right and left expansion optics are configured to receive respective display images from the right and left display projectors and to release expanded forms of the display images. The right IMU is fixedly coupled to the right display projector, and the left IMU is fixedly coupled to the left display projector. Each ASP element is responsive to an angle of light of one of the respective display images as received into the right or left expansion optic. The computer is configured to receive output from the right IMU, the left IMU and the plurality of ASP elements, and render display data for the right and left display projectors based in part on the output.

Autocalibrated near-eye display

A computer-implemented method of field calibrating a device is disclosed. In one example, a reference image is acquired, via an infrared camera, while the device is in a factory-calibrated state. The reference image measures a factory-calibrated spatial relationship of one or more infrared-visible fiducial markers of the device relative to the infrared camera while the device is in the factory-calibrated state. Later, a field image is acquired, via the infrared camera. The field image measures an updated spatial relationship of the one or more infrared-visible fiducial markers relative to the infrared camera. The device is field calibrated based on the reference image and the field image.

Fiducial marker based field calibration of a device

Examples are disclosed that relate to displaying a hologram via an HMD. One disclosed example provides a method comprising obtaining depth data from a direct-measurement depth sensor included in the case for the HMD, the depth data comprising a depth map of a real-world environment. The method further comprises determining a distance from the HMD to an object in the real-world environment using the depth map, obtaining holographic imagery for display based at least upon the distance, and outputting the holographic imagery for display on the HMD.

Depth sensing via device case

The techniques disclosed herein detect sensor misalignments in a display device by the use of sensors operating under different modalities. In some configurations, a near-to-eye display device can include a number of sensors that can be used to track movement of the device relative to a surrounding environment. The device can utilize multiple sensors operating under multiple modalities. For each sensor, there is a set of intrinsic and extrinsic properties that are calibrated. The device is also configured to determine refined estimations of the intrinsic and extrinsic properties at runtime. The refined estimations of the intrinsic and extrinsic properties can then be used to derive knowledge on how the device has deformed over time. The device can then use the refined estimations of the intrinsic and extrinsic properties and/or any other resulting data that quantifies any deformations of the device to make adjustments to rendered images at runtime.

Growing community of inventors

Redmond, WA, United States of America

Rayna Demaster-Smith