Rockwell Collins, Inc.

William T Kirchner

Christopher M Boggs

Brandon E Wilson

Gavin P Haentjens

Vladislav Gavrilets

Troy D Driscoll

Keith M Allen

Ramachandra J Sattigeri

Kurt I Jaeger

Christopher Alan Keith

Eric P Stratton

James H Doty

Weston James Lahr

Bobby D Foote

David I Han

Christopher L George

Alexander D Reid

Ankur Ankur

Allen David Wu

Gregory J Kelsey

Yingyue Xu

Eric Brewer

A system includes a head wearable display (HWD) device and a processor. The HWD device includes a waveguide display and an image monitor sensor. The waveguide display includes a waveguide and an optical system, the optical system configured to: receive image data corresponding to an image; and project the image at least through the waveguide to be displayed to a user. The image monitor sensor is optically coupled to the waveguide or the optical system. The image monitor sensor is configured to: capture at least a portion of the image as a monitored image; and output monitored image data. The processor is configured to: receive the monitored image data from the image monitor sensor; and based at least on the monitored image data, determine whether the waveguide display displayed the image correctly.

Head wearable display device with image monitoring comprising at least one image monitor sensor optically coupled to a waveguide

A system and method for high integrity head pose estimation receives a 2D image including medium-density fiducials and low-density fiducials. The system then detects the medium-density fiducials and compares them with a 3D constellation to form an initial head pose estimate. The initial head pose estimate is then checked for possible alternative correspondences. A selection from a set of candidate initial head pose estimates is made to determine. The selected head pose estimate then has search areas established for the medium-density fiducials and low-density fiducials. The system then determines whether or not the head pose is valid and accurate based on whether or not the fiducials appear in their respective search areas.

High-confidence optical head pose correspondence mapping with multiple lower-density markers for high-integrity headtracking on a headworn display (HWD)

A system and method for modifying symbology displayed on a head worn display (HWD) based on a quality-of-service value is disclosed. The system includes a head worn display configured to display symbology, a tracking camera, a controller, and a tracking processing unit configured to receive inputs form the tracking camera and transmit a signal quality value to the controller. The controller includes processors and memory with instructions that when executed by the processors, cause the processors to display the symbology on the display, determine a quality-of-service value based on at least the signal quality value, and modify at least one symbol of the symbology based on the quality-of-service value from a default symbol to a degraded symbol if the quality-of-service value changes from a high-quality value to a low-quality value. The system may also receive sensor signals from other sensors, and use the sensor signal to determine the quality-of-service value.

Accuracy adaptive symbology for head worn display

A system and method for enhanced aircraft-based targeting senses RF emissions or other signals associated with a target while navigating a trajectory through a GPS-challenged airspace. While the target is being observed, the aircraft targeting system tracks GPS-challenged state vectors (e.g., via an onboard inertial reference system) and pressure altitudes consistent with each observation. When the aircraft emerges from the GPS-challenged airspace, the targeting system determines multiple GPS-driven subsequent absolute positions of the aircraft. The targeting system determines a refined estimate of the target location via batch factor graph optimization of measurements taken while inside and outside of the GPS-challenged airspace.

System and method for navigation and targeting in GPS-challenged environments using factor graph optimization

A system and method established and maintains precision relative position, navigation, and timing (PNT) across a network of at least four mutually connected mobile platforms. In embodiments, a key (e.g., advantaged, absolute positioning capable) node of the network determines its pressure altitude and inertial state relative to its platform reference frame and receives inertial state and pressure altitude data from each neighboring node (in exchange for its own) to estimate the relative position and orientation of each neighbor node in its platform frame. The key node performs ranging to each neighboring node, and the neighboring nodes additionally range between each other and exchange ranging data with the key node. By correcting position and orientation estimates via ranging data, the key node determines and maintains extended relative PNT (e.g., in GPS-denied areas), which relative PNT solution is distributed across all network nodes.

System and method for maintaining cooperative precision navigation and timing (PNT) across networked platforms in contested environments

A system and method for head tracking for a head worn display (HWD) includes head-frame and platform-frame IMUs providing high-rate pose data of the wearer's head and a mobile platform. An optical tracker estimates an optical pose of a camera oriented at fiducials based on a comparison of 2D image data captured by the camera (and portraying the fiducials within the image) and the known location of each fiducial in a 3D marker frame. The pIMU also provides low-rate, high-integrity georeferenced pose data of the mobile platform (e.g., in an earth frame). The head tracker provides low-rate, high-integrity absolute head pose solutions (e.g., user head pose in the earth frame) based on the georeferenced pose data and current high-rate head/platform pose data, updating the absolute pose solutions with high-rate updates based on combined extended Kalman filter propagation of the high-rate head and platform pose data with refined optical pose data.

Head tracking system with extended kalman filter combining absolute and relative navigation

A system and method for conformal alignment of an aircraft-based head worn display (HWD) system defines far-field features in view of the HWD display, each far-field feature having a 3D truth position in a navigational reference frame. The aircraft GPS/IRS determines pointing vectors to each feature in the aircraft reference frame. The HWD user estimates a nominal head pose for orienting the far-field features in the HWD display, and the HWD system renders the far-field features based on the nominal head pose and pointing vectors. The rendered far-field features are aligned to their real-world 3D truth positions (either manually or with the assistance of witness cameras) and a latched pose of the HWD headtracker system determined based on a successful alignment within accuracy bounds. Based on the latched headtracker pose, the HWD headtracker alignment and the alignment of the HWD display reference frame to the aircraft reference frame are updated.

System and method for conformal head worn display (HWD) headtracker alignment

A system and method for predicting an estimated head pose solution of a head worn display (HWD) at a future time selects or determines, for each HWD rotational axis, a relative angular rate of a head reference frame relative to a body reference frame. By sampling the relative angular rate, a relative angular acceleration is determined. Based on a projection time associated with estimated display latency, angular increment vectors for each rotational axis are determined for relative angular rate and relative angular acceleration. The current head pose solution is adjusted with respect to one or more axes based on the angular increment vectors to project the head pose estimate ahead based on the projection time, the resulting projected head pose solution corresponding to a predicted orientation of the head frame relative to the body frame at the future time.

System and method for pose prediction in head worn display (HWD) headtrackers

A head tracking system uses coded features, highly structured compared to the operating environment, to ensure a high-integrity correspondence map. Coded features can be used to provide a negligible probability of spurious features and probability of misidentification. Ensuring a reliable correspondence map prevents unmodeled errors that arise from an invalid correspondence map. In the case of multiple outliers, existing screening techniques, such as random sample consensus (RANSAC) or fault exclusion, may be used to eliminate excessive outliers. Mature GPS integrity techniques may then be extended to optical pose estimation to establish integrity bounds for single faults that may go undetected by fault detection.

High-integrity optical pose estimation using coded features

A collision avoidance system includes a processor configured to determine first and second positions of an intruder in first and second images, respectively, determine an angular position change of the intruder by comparing the first and second positions, determine a rate of change of a line of sight angle from an aircraft to the intruder based on the angular position change and a time between the first and second images, determine a rate of change of the angular size of the intruder based on a difference in angular size of the intruder from the first to second image and the time between the first and second images, and provide an alert to an operator of the aircraft in response to the ratio of the rate of change of the line of sight angle and the rate of change of the angular size of the intruder being less than a threshold.

Midair collision threat detection and assessment using visual information

A system and related method for hybrid headtracking receives head-referenced pose data from a head-mounted IMU and platform-referenced or georeferenced position and orientation (pose) data from a platform-mounted IMU, determines error models corresponding to uncertainties associated with both IMUs, and performs an initial estimate of head pose relative to the platform reference frame based on the head-referenced and platform-referenced pose data. A numerically stable UD factorization of the Kalman filter propagates the estimated head pose data forward in time and corrects the initial head pose estimate (and may correct the error models associated with the head-mounted and platform-mounted IMUs) based on secondary head pose estimates, and corresponding error models, received from an optical or magnetic aiding device. The corrected head pose data is forward to a head-worn display to ensure high accuracy of displayed imagery and symbology.

System and method for hybrid optical/inertial headtracking via numerically stable Kalman filter

A system and method for automatic boresighting of a head-tracking system for a HWD or HUD accesses georeferenced pose (position/orientation) data of an aircraft or other mobile platform from a georeferenced head-tracker and platform-referenced head pose data (based on fiducial markers within the aircraft cockpit) from the inertial measurement units of an optical, magnetic, or optical/magnetic platform-referenced head tracker (PRHT) and determines misalignments of the PRHT relative to the boresighted inertial reference system (IRS) of the aircraft. Based on the determined misalignments and accessed data, the IMUs of the PRHT may be updated and aligned with the aircraft boresight. Misalignments of head-mounted fiducial markers with the boresighted IRS may be determined for updating the alignment of an optical head-tracker fixed relative to the aircraft cockpit.

Automatic boresighting of head-worn display

A registration solution between two or more sets of point cloud data is provided to register a first target set of point cloud data with a second source set of point cloud data. Through error propagation, degrees of freedom masking and transformation filtering, a system may provide an intelligent method of enabling and disabling degrees of freedom in a real-time registration application. A system may adapt to varying environments (feature-rich and featureless) ensuring a high-integrity registration estimate through providing estimates of the observable parameters. By optionally and iteratively masking zero or more degrees of freedom from the registration estimate, a system may locate desirable data to use in a registration estimate. Then, the system may blend updates from the registration estimates to achieve the best available registration solution applied to the original second source set as well as additional sets of point cloud data.

Growing community of inventors

Leesburg, VA, United States of America

William T Kirchner