Enhanced detection of sensor misalignments by coordinating sensors utilizing multiple sensing modalities

Abstract

The techniques disclosed herein detect sensor misalignments in a display device by the use of sensors operating under different modalities. In some configurations, a near-to-eye display device can include a number of sensors that can be used to track movement of the device relative to a surrounding environment. The device can utilize multiple sensors operating under multiple modalities. For each sensor, there is a set of intrinsic and extrinsic properties that are calibrated. The device is also configured to determine refined estimations of the intrinsic and extrinsic properties at runtime. The refined estimations of the intrinsic and extrinsic properties can then be used to derive knowledge on how the device has deformed over time. The device can then use the refined estimations of the intrinsic and extrinsic properties and/or any other resulting data that quantifies any deformations of the device to make adjustments to rendered images at runtime.