The Boeing Company

Jacob A Lucas

Justin Cleve Hatcher

Trent M Kyono

James L Clayton

Steven Fulton Griffin

Paul S Idell

Yifan Yang

Nicole Catherine Gagnier

Jason J Rock

Alexander C Klein

A method includes providing an image to a feature extraction model to generate feature data. The image depicts a portion of a first device and a portion of a second device. The feature data includes coordinates representing key points of each of the first and second devices depicted in the image. The method also includes obtaining position data indicating a position in 3D space of a connector of the first device. The method further includes providing the feature data and the position data to a trained autonomous agent to generate a proposed maneuver to mate the connector with a connector of the second device.

Autonomous maneuver generation to mate connectors

Aspects of the disclosure provide solutions for determining a position of an object in a video frame. Examples include: receiving a segmentation mask of an identified object in a video frame; adjusting a 3D representation of a moveable part of the object based on constraints for the moveable part; comparing the 3D model of the object to the segmentation mask of the object; determining a match between the 3D model of the object to the segmentation mask of the object is above a threshold; and based on the match being above the threshold, determining a position of the object.

Pose detection of an object in a video frame

Aspects of the disclosure provide solutions for automated air-to-air refueling (A3R) and assisted air-to-air refueling. Examples include: receiving a video frame; generating, from the video frame, a plurality of images having differing decreasing resolutions; detecting, within each of the plurality of images, a set of aircraft keypoints for an aircraft to be refueled; merging the sets of aircraft keypoints into a set of merged aircraft keypoints; based on at least the merged aircraft keypoints, determining a position of a fuel receptacle on the aircraft; and determining a position of a boom tip of an aerial refueling boom. Some examples include, based on at least the position of the fuel receptacle and the position of the boom tip, controlling the aerial refueling boom to engage the fuel receptacle, and for some examples, the video frame is monocular (e.g., provided by a single camera).

Feature detection for automated air-to-air refueling (A3R)

Aspects of the disclosure provide solutions for automated air-to-air refueling (A3R) and assisted air-to-air refueling. Examples include: receiving a video frame; identifying an aircraft to be refueled from within the video frame; segmenting the video frame to generate a segmentation mask matching the aircraft within the video frame; based on at least the segmentation mask, determining a position of a fuel receptacle on the aircraft; determining a position of a boom tip of the aerial refueling boom; and controlling the aerial refueling boom to engage the fuel receptacle based on at least the position of the fuel receptacle and the position of the boom tip.

Segmentation-based fuel receptacle localization for air-to-air refueling (A3R)

An example system includes a processor and a non-transitory computer-readable medium having stored therein instructions that are executable to cause the system to perform various functions. The functions include: (i) acquiring an image of a first aerial vehicle, the image depicting an object of a second aerial vehicle prior to contact between the object and a surface of the first aerial vehicle; (ii) providing the image as input to a data-driven analyzer that is trained in a supervised setting with example images for determining a predetermined point of contact between the object and the surface of the first aerial vehicle; (iii) determining, based on an output of the data-driven analyzer corresponding to the input, an estimated point of contact between the object and the surface of the first aerial vehicle; and (iv) providing the estimated point of contact to a display system.

Virtual laser pointer as a point of contact indicator for machine learning assisted aerial refueling or other targeting

A spin-stabilizing assembly is usable with a cylindrical barrel having a longitudinal center axis, a vertical center axis, and a distal end. The assembly includes a gimbal piece, a spinner cage, and a spinner body. A first end of the gimbal piece engages an outer diameter surface of the barrel proximate the distal end. The spinner cage is configured to engage a second end of the gimbal piece, and to pivot with respect thereto about the vertical center axis. The spinner body is disposed radially within the spinner cage and defines multiple axial vanes. The vanes, in response to impingement on the axial vanes of exhaust gases discharged from the distal end of the barrel, rotate the spinner body about the longitudinal center axis. Rotation results in impedance along the vertical center axis which minimizes vertical displacement or motion of the distal end.

Growing community of inventors

Makawao, HI, United States of America

Jacob A Lucas