Rockwell Collins, Inc.

Rockwell Collins Control Technologies, Inc.

Christopher M Boggs

William T Kirchner

Brandon E Wilson

Gavin P Haentjens

Weston James Lahr

Troy D Driscoll

Richard M Rademaker

Max G Taylor

Keith M Allen

Ramachandra J Sattigeri

Arjun Passi

Robert B Anderson

Kurt I Jaeger

Christopher Alan Keith

Eric P Stratton

Vladislav Gavrilets

Michael D Piedmonte

David W Vos

Alexander D Reid

Bryan M Krawiec

Ankur Ankur

Gregory E Mott

Jack Jordan

Joel J Fuster

Gregory J Kelsey

Yingyue Xu

Paxton Alsgaard

Amy Lindaman

John D Anderson

David L Lempia

Vadislav Gavrilets

A system includes a head wearable display (HWD) device and a processor. The HWD device includes a waveguide display and an image monitor sensor. The waveguide display includes a waveguide and an optical system, the optical system configured to: receive image data corresponding to an image; and project the image at least through the waveguide to be displayed to a user. The image monitor sensor is optically coupled to the waveguide or the optical system. The image monitor sensor is configured to: capture at least a portion of the image as a monitored image; and output monitored image data. The processor is configured to: receive the monitored image data from the image monitor sensor; and based at least on the monitored image data, determine whether the waveguide display displayed the image correctly.

Head wearable display device with image monitoring comprising at least one image monitor sensor optically coupled to a waveguide

A system and method for high integrity head pose estimation receives a 2D image including medium-density fiducials and low-density fiducials. The system then detects the medium-density fiducials and compares them with a 3D constellation to form an initial head pose estimate. The initial head pose estimate is then checked for possible alternative correspondences. A selection from a set of candidate initial head pose estimates is made to determine. The selected head pose estimate then has search areas established for the medium-density fiducials and low-density fiducials. The system then determines whether or not the head pose is valid and accurate based on whether or not the fiducials appear in their respective search areas.

High-confidence optical head pose correspondence mapping with multiple lower-density markers for high-integrity headtracking on a headworn display (HWD)

A system and method for modifying symbology displayed on a head worn display (HWD) based on a quality-of-service value is disclosed. The system includes a head worn display configured to display symbology, a tracking camera, a controller, and a tracking processing unit configured to receive inputs form the tracking camera and transmit a signal quality value to the controller. The controller includes processors and memory with instructions that when executed by the processors, cause the processors to display the symbology on the display, determine a quality-of-service value based on at least the signal quality value, and modify at least one symbol of the symbology based on the quality-of-service value from a default symbol to a degraded symbol if the quality-of-service value changes from a high-quality value to a low-quality value. The system may also receive sensor signals from other sensors, and use the sensor signal to determine the quality-of-service value.

Accuracy adaptive symbology for head worn display

A system and method for high-confidence error overbounding of multiple optical pose solutions receives a set of candidate correspondences between 2D image features captured by an aircraft camera and 3D constellation features including at least one ambiguous correspondence. A candidate estimate of the optical pose of the camera is determined for each of a set of candidate correspondence maps (CMAP), each CMAP resolving the ambiguities differently. Each candidate pose estimate is evaluated for viability and any non-viable estimates eliminated. An individual error bound is determined for each viable candidate pose estimate and CMAP, and based on the set of individual error bounds a multiple-pose containment error bound is determined, bounding with high confidence the set of candidate CMAPs and multiple pose estimates where at least one is correct. The containment error bound may be evaluated for accuracy as required for flight operations performed by aircraft-based instruments and systems.

Vision-based navigation system incorporating high-confidence error overbounding of multiple optical poses

A vision-based navigation system (e.g., for aircraft on approach to a runway) captures via camera 2D images of the runway environment in an image plane. The vision-based navigation system stores a constellation database of runway features and their nominal 3D position information in a constellation plane. Image processors detect within the captured images 2D features potentially corresponding to the constellation features. The vision-based navigation system estimates optical pose of the camera in the constellation plane by aligning the image plane and constellation plane into a common domain, e.g., via orthocorrection of detected image features into the constellation plane or reprojection of constellation features into the image plane. Based on the common-domain plane, the vision-based navigational system generates candidate correspondence maps (CMAP) of constellation features mapped to the image features with high-confidence error bounding, from which optical pose of the camera or aircraft can be estimated.

Vision-based navigation system incorporating model-based correspondence determination with high-confidence ambiguity identification

A system and method utilizing an existing boresighted HUD and a camera to determine the transformational and rotational relationships between the camera image and the boresighted HUD image of an aircraft to properly orient the camera for landing operations. Known boresight features are identified in a HUD image and are related to the same boresight features in a camera image, and transformations are applied to align pixels in the camera image to the pixel locations of the boresight features in the HUD image. Object recognition algorithms may be employed to identify the boresight features in the camera image.

Aircraft camera extrinsic calibration using head-up display

Convertible smokeless fire pit

A system and method for head tracking for a head worn display (HWD) includes head-frame and platform-frame IMUs providing high-rate pose data of the wearer's head and a mobile platform. An optical tracker estimates an optical pose of a camera oriented at fiducials based on a comparison of 2D image data captured by the camera (and portraying the fiducials within the image) and the known location of each fiducial in a 3D marker frame. The pIMU also provides low-rate, high-integrity georeferenced pose data of the mobile platform (e.g., in an earth frame). The head tracker provides low-rate, high-integrity absolute head pose solutions (e.g., user head pose in the earth frame) based on the georeferenced pose data and current high-rate head/platform pose data, updating the absolute pose solutions with high-rate updates based on combined extended Kalman filter propagation of the high-rate head and platform pose data with refined optical pose data.

Head tracking system with extended kalman filter combining absolute and relative navigation

A hierarchical modular arbitration architecture for a mobile platform guidance system is disclosed. In embodiments, the architecture comprises a hierarchy of arbitration layers, each arbitration layer narrower in scope than the layer above (e.g., mission objective arbitrators, route arbitrators, path arbitrators). Each arbitration layer includes one or more objective-based arbitrators in communication with one or more applications or modes. Each arbitrator receives control input (e.g., from the pilot, from aircraft sensors) and control signals from the level above, selecting a mode to make active based on decision agents within the arbitrator layer which control mode priorities and sequencing (e.g., some flight objectives may involve multiple arbitrators and their subject applications coordinating in sequence). Each arbitrator passes control signals associated with fulfilling the commands of the active mode to the level below and reports application and error information to the arbitrator level above and/or human/artificial pilot machine interfaces.

Platform guidance system incorporating hierarchical modular arbitration architecture

A system and method for conformal alignment of an aircraft-based head worn display (HWD) system defines far-field features in view of the HWD display, each far-field feature having a 3D truth position in a navigational reference frame. The aircraft GPS/IRS determines pointing vectors to each feature in the aircraft reference frame. The HWD user estimates a nominal head pose for orienting the far-field features in the HWD display, and the HWD system renders the far-field features based on the nominal head pose and pointing vectors. The rendered far-field features are aligned to their real-world 3D truth positions (either manually or with the assistance of witness cameras) and a latched pose of the HWD headtracker system determined based on a successful alignment within accuracy bounds. Based on the latched headtracker pose, the HWD headtracker alignment and the alignment of the HWD display reference frame to the aircraft reference frame are updated.

System and method for conformal head worn display (HWD) headtracker alignment

A system and method for predicting an estimated head pose solution of a head worn display (HWD) at a future time selects or determines, for each HWD rotational axis, a relative angular rate of a head reference frame relative to a body reference frame. By sampling the relative angular rate, a relative angular acceleration is determined. Based on a projection time associated with estimated display latency, angular increment vectors for each rotational axis are determined for relative angular rate and relative angular acceleration. The current head pose solution is adjusted with respect to one or more axes based on the angular increment vectors to project the head pose estimate ahead based on the projection time, the resulting projected head pose solution corresponding to a predicted orientation of the head frame relative to the body frame at the future time.

System and method for pose prediction in head worn display (HWD) headtrackers

A system and method for identifying a safe path in a partially unknown environment in a time critical fashion includes identifying long range paths that avoid known obstacles. Short range paths within a receding horizon are identified corresponding to the long range paths, the short range paths avoiding known obstacles and unknown regions. Each short range path is broken into segments based on subsequent control operations or timing, and abort procedures are identified corresponding to each segment to enforce time critical availability of a safe action. Short range paths are excluded unless abort procedures can be identified for subsequent segments.

Time-critical obstacle avoidance path planning in uncertain environments

A method for occupancy map synchronization in multi-vehicle networks may include receiving three-dimensional perception sensor (3DPS) data. The method may include generating a data message including an occupancy map (OM) octree. The OM octree may be generated from an occupancy map constructed from the 3DPS data. The method may include reducing at least one of a messaging time or a messaging size of the data message including the OM octree. The method may include transmitting the data message including the OM octree via a leader vehicle controller communicatively coupled to a leader vehicle. The method may include receiving the data message comprising the OM octree via a follower vehicle controller communicatively coupled to the follower vehicle. The method may include generating a local occupancy map from the data message comprising the OM octree via the follower vehicle controller.

Occupancy map synchronization in multi-vehicle networks

A system in a mobile platform creates a three-dimensional world model from sets of volume elements organized into sets of volume elements of increasing size and converted into an octree data structure. The system receives a request for a map of a specific, limited region and at a specific, limited level of detail and produces a reduced three-dimensional world model to conform to the limited region and level of detail. The reduced map is transmitted to the requesting platform and periodically updated. The map may be reduced by categorizing the lowest level of volume elements according to the worst case of any constituent volume elements. Alternatively, the system may indicate in the lowest level of volume elements whether the receiving platform would benefit from an increased level of detail.

Efficient transfer of dynamic 3D world model data

A head tracking system uses coded features, highly structured compared to the operating environment, to ensure a high-integrity correspondence map. Coded features can be used to provide a negligible probability of spurious features and probability of misidentification. Ensuring a reliable correspondence map prevents unmodeled errors that arise from an invalid correspondence map. In the case of multiple outliers, existing screening techniques, such as random sample consensus (RANSAC) or fault exclusion, may be used to eliminate excessive outliers. Mature GPS integrity techniques may then be extended to optical pose estimation to establish integrity bounds for single faults that may go undetected by fault detection.

Growing community of inventors

Gainesville, VA, United States of America

Christopher M Boggs