The Boeing Company

Shawn M Chamberlain

Deepak Khosla

Justin Cleve Hatcher

Jung Soon Jang

Leon Nguyen

Fan Hin Hung

Haden Harrison Smith

Daniel Michael O'Shea

Luis-Alberto J Santiago

Joshua Neighbor

Disclosed herein is methods, systems, and aircraft for performing image analysis for aiding refueling operations. A tanker aircraft includes a single camera, a refueling boom, a camera configured to generate a 2D image of the refueling boom, a processor, and non-transitory computer readable storage media storing code. The code being executable by the processor to perform operations comprising receiving the 2D image from the single camera, determining 2D keypoints of the refueling boom located within the 2D image based on a predefined point model of the refueling boom, determining keypoints in 3D space based on the 2D keypoints to produce 3D keypoints, determining a 6DOF pose using the 2D keypoints and the 3D keypoints, and estimating a position of a tip of the refueling boom based on the 6DOF pose.

Systems and methods for calculating refueling tanker boom tip 3D position for aerial refueling

A trajectory planning system for a refueling boom includes a human-machine interface ('HMI') device and an electronic control unit (“ECU”). The HMI device outputs electronic control signals, in response to which the ECU performs a method. The ECU accesses a three-dimensional (“3D”) boundary model of the receiver and a 3D model of the boom. The ECU calculates a boom-to-receiver relative position using the models and sensor data, and a planned trajectory between a boom tip and a receptacle on the receiver. The trajectory is calculated using the boom-to-receiver relative position and predictive artificial potential fields. A point is found on a baseline trajectory farthest from a straight line between the receptacle and boom tip, which is recorded as a temporary goal. The planned trajectory avoids contact between the boom and receiver features. The ECU executes a control action using the planned trajectory.

Artificial potential field-based boom guidance for automated air-to-air refueling

A system for calibrating a camera for use aboard a fuel-supplying aircraft ('tanker') for an automated air-to-air refueling (“A3R”) process in which the tanker offloads aviation fuel to a fuel-receiving aircraft (“receiver”) includes an electronic control unit having a processor and a non-transitory computer-readable storage medium on which is recorded instructions for calibrating the camera. Executing the instructions causes the processor to perform a corresponding method, i.e., to receive boom position data indicative a true position of the boom in free space, and real-time image data from the camera. The instructions also cause the processor to use the boom position data and a 3D model of the boom to project a digital representation of the boom into the image space. The processor then minimizes a difference between the projected and real images of the boom to thereby calibrate the camera.

Camera calibration optimization using image segmentation and boom resolvers for automated air-to-air refueling

Growing community of inventors

Federal Way, WA, United States of America

Shawn M Chamberlain