Rgbd-based neural network and training for pose estimation

Abstract

Disclosed herein are methods, systems, and aircraft for performing image analysis for aiding refueling operations. A tanker aircraft includes a camera system configured to generate a two-dimensional (2D) image of a receiving aircraft and a depth image of the receiving aircraft, a processor, and non-transitory computer readable storage media storing code. The code is executable by the processor to perform operations including detecting the receiving aircraft within the 2D image based on 2D image values, determining 2D keypoints of the receiving aircraft located within the 2D image based on the depth image, the 2D image values, and a previously determined keypoint model, determining a 6 degree-of-freedom (6DOF) pose using the 2D keypoints and corresponding three-dimensional (3D) keypoints, and outputting a position of at least a component of the receiving aircraft based on the 6DOF pose.