US 12,087,017 B2
Camera calibration optimization using image segmentation and boom resolvers for automated air-to-air refueling
Joshua Neighbor, Seattle, WA (US); Shawn M. Chamberlain, Federal Way, WA (US); and Luis-Alberto J. Santiago, Edmonds, WA (US)
Assigned to The Boeing Company, Arlington, VA (US)
Filed by The Boeing Company, Arlington, VA (US)
Filed on Jan. 20, 2023, as Appl. No. 18/157,341.
Prior Publication US 2024/0249437 A1, Jul. 25, 2024
Int. Cl. G06T 7/80 (2017.01); B64D 39/00 (2006.01); B64D 43/00 (2006.01); B64D 47/08 (2006.01); G06T 7/136 (2017.01); G06T 7/194 (2017.01)
CPC G06T 7/80 (2017.01) [B64D 39/00 (2013.01); B64D 43/00 (2013.01); B64D 47/08 (2013.01); G06T 7/136 (2017.01); G06T 7/194 (2017.01)] 20 Claims
OG exemplary drawing
 
1. A system for calibrating a camera for use aboard a fuel-supplying aircraft (“tanker”) for an automated air-to-air refueling (“A3R”) process, the system comprising:
a processor; and
a non-transitory computer-readable storage medium (“memory”) on which is recorded a three-dimensional (“3D”) model of a refueling boom and instructions for calibrating the camera, wherein the 3D model of the refueling boom includes a digital representation of the refueling boom, and wherein execution of the recorded instructions causes the processor to:
receive boom position data from a boom resolver, the boom position data being indicative of a true position of the refueling boom in free space;
receive real-time image data of the refueling boom from the camera in an image space of the camera as a real image;
project the digital representation of the refueling boom into the image space as a projected image using the boom position data and a boom-to-camera (“bTc”) relationship; and
iteratively calculate an optimized value of the bTc relationship to minimize a value of an objective function until a difference between a position of the real image and a position of the projected image is minimized within the image space to thereby calibrate the camera, including iteratively calculating the optimized value of the bTc relationship until a Dice similarity coefficient (“Dice”) is greater than about 0.9, wherein the objective function is 1-Dice.