US 12,406,375 B2
Pose detection of an object in a video frame
Jason J. Rock, Tukwila, WA (US); Jacob A. Lucas, Kihei, HI (US); Trent M. Kyono, Kihei, HI (US); and Justin C. Hatcher, Tukwila, WA (US)
Assigned to The Boeing Company, Arlington, VA (US)
Filed by The Boeing Company, Chicago, IL (US)
Filed on Jun. 24, 2022, as Appl. No. 17/849,591.
Claims priority of provisional application 63/225,935, filed on Jul. 26, 2021.
Prior Publication US 2023/0026394 A1, Jan. 26, 2023
Int. Cl. G06T 7/11 (2017.01); G06T 7/73 (2017.01); G06V 20/17 (2022.01)
CPC G06T 7/11 (2017.01) [G06T 7/75 (2017.01); G06V 20/17 (2022.01); G06T 2207/10016 (2013.01); G06T 2207/10032 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system for determining a position of an aerial refueling boom from a video frame, the system comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving the video frame;
identifying the aerial refueling boom in the video frame, the aerial refueling boom comprising a rigid part and a moveable part;
generating a segmentation mask of the aerial refueling boom;
accessing a three-dimensional (3D) model of the aerial refueling boom, the 3D model of the aerial refueling boom comprising constraints for the moveable part of the aerial refueling boom,
wherein the constraints include information on limitations of how the moveable part may move;
making a first comparison of the 3D model of the aerial refueling boom to the segmentation mask of the aerial refueling boom;
based on the first comparison, determining a match between the 3D model of the aerial refueling boom to the segmentation mask of the aerial refueling boom is below a threshold;
based on determining the match between the 3D model of the aerial refueling boom to the segmentation mask of the aerial refueling boom is below the threshold, using the constraints to adjust a 3D representation of the moveable part in the 3D model of the aerial refueling boom,
wherein the adjusting is performed using gradient descent;
upon adjusting the 3D representation of the moveable part in the 3D model of the aerial refueling boom, making a second comparison of the 3D model of the aerial refueling boom to the segmentation mask of the aerial refueling boom;
based on the second comparison, determining the match between the 3D model of the aerial refueling boom to the segmentation mask of the aerial refueling boom is above the threshold;
based on determining the match between the 3D model of the aerial refueling boom to the segmentation mask of the aerial refueling boom is above the threshold, determining the position of the aerial refueling boom; and
controlling, based on determining the position of the aerial refueling boom, the aerial refueling boom to engage a fuel receptacle.