| CPC G06T 7/12 (2017.01) [G06T 7/20 (2013.01); G06V 10/44 (2022.01); G06V 10/60 (2022.01); G06V 20/17 (2022.01)] | 5 Claims |
|
1. A method for detecting and repairing sun glint in Unmanned Aerial Vehicle (UAV) optical RGB ocean images, which comprises: constructing a semantic segmentation network model based on the attention mechanism by using the sun glint attention module; guiding the semantic segmentation network training by a hybrid loss function of Focal and Dice, detecting the sun glint region by the trained semantic segmentation network structure (SGNet); extracting an optical flow field of high-resolution UAV optical RGB ocean images by RAFT optical flow estimation network, and transmitting the sun glint region by optical flow propagation to repair the sun glint in UAV optical RGB ocean images and recover the real benthic image features;
The method for detecting and repairing sun glint in UAV optical RGB ocean images comprises the following steps:
Step 1, introducing the sun glint attention (SGA) module into a Convolutional Networks for Biomedical Image Segmentation (UNet network structure) to extract and enhance the interesting sun glint features and construct the semantic segmentation network;
Step 2, guiding the semantic segmentation network training by the Focal and Dice hybrid loss function, and detecting the sun glint region by the trained semantic segmentation network SGNet;
Step 3, extracting the optical flow fields of adjacent high-resolution UAV optical RGB ocean images by Recurrent All-Pairs Field Transforms (RAFT) optical flow estimation network;
Step 4, propagating optical flow between image frames by the sun glint region detected by SGNet and the optical flow field extracted by RAFT to repair the pixels blocked by sun glint;
Step 5, repairing a single image by a coherent semantic attention (CSA) image generation network for pixels that cannot be repaired by optical flow propagation, and adding a repair result to the optical flow propagation of the next iteration as a known value to obtain the final sun glint repair result;
Optical flow propagation between image frames in Step 4: obtaining forward and backward optical flow propagation of pixels blocked by sun glint respectively until two known pixels; calculating consistency errors of two known pixels respectively, and assigning the weights according to the errors; Finally, obtaining the final repaired pixels by weighted fusion of two known pixels; Among them, the formula for calculating the consistency error is as follows:
 Wherein, i and j represent the corresponding pixels on image frame m and image frame n, errmn represents consistency errors, and fmn represents the optical flow value from image frame m to image frame n;
The weighted fusion weight formula is as follows:
 Wherein, w represents the weight, err represents the consistency error, and errmean represents the mean value of the consistency error.
|