The Boeing Company

Joshua Neighbor

Shawn M Chamberlain

Luis-Alberto J Santiago

An automated air-to-air refueling (A3R) system is usable with a tanker aircraft having a refueling boom. The system includes a camera connected to the tanker in proximity to the refueling boom which outputs a video stream of the boom and a fuel-receiving aircraft/receiver during an aerial refueling process. The system also includes a human-machine interface ('HMI') located aboard the tanker, and an electronic control unit (“ECU”) in communication with the camera and HMI. The ECU identifies keypoints on the receiver indicative of flight control surfaces thereof, tracks corresponding positions of the flight control surfaces in real-time, and predicts a change in position of the receiver in free space as a predicted 3D position using the corresponding positions. The HMI also outputs a directional indicator indicative of the predicted 3D position, e.g., as a graphical overlay to a display screen.

Automated air-to-air refueling using keypoint-based perception of aircraft flight control surfaces

A system for calibrating a camera for use aboard a fuel-supplying aircraft ('tanker') for an automated air-to-air refueling (“A3R”) process in which the tanker offloads aviation fuel to a fuel-receiving aircraft (“receiver”) includes an electronic control unit having a processor and a non-transitory computer-readable storage medium on which is recorded instructions for calibrating the camera. Executing the instructions causes the processor to perform a corresponding method, i.e., to receive boom position data indicative a true position of the boom in free space, and real-time image data from the camera. The instructions also cause the processor to use the boom position data and a 3D model of the boom to project a digital representation of the boom into the image space. The processor then minimizes a difference between the projected and real images of the boom to thereby calibrate the camera.

Growing community of inventors

Seattle, WA, United States of America

Joshua Neighbor