US 12,423,786 B2
Multi-scale fusion defogging method based on stacked hourglass network
Dengyin Zhang, Nanjing (CN); Qian Zhao, Nanjing (CN); and Jingyu Wang, Nanjing (CN)
Assigned to NANJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS, Nanjing (CN)
Filed by NANJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS, Nanjing (CN)
Filed on May 4, 2023, as Appl. No. 18/312,168.
Application 18/312,168 is a continuation of application No. PCT/CN2023/086215, filed on Apr. 4, 2023.
Claims priority of application No. 202211007029.3 (CN), filed on Aug. 22, 2022.
Prior Publication US 2024/0062347 A1, Feb. 22, 2024
Int. Cl. G06T 5/80 (2024.01); G06T 3/40 (2024.01)
CPC G06T 5/80 (2024.01) [G06T 3/40 (2013.01); G06T 2207/20016 (2013.01); G06T 2207/20084 (2013.01)] 5 Claims
OG exemplary drawing
 
1. A multi-scale fusion defogging method based on a stacked hourglass network, comprising:
inputting a foggy image into a preset image defogging network; and
outputting a fogless image after the foggy image is processed by the image defogging network;
wherein the image defogging network comprises a 7×7 convolutional layer, a stacked hourglass module, a feature fusion, a multi-scale jump connection module, a 1×1 convolutional layer, a 3×3 convolutional layer, a hierarchical attention distillation module, the 3×3 convolutional layer and the 1×1 convolutional layer connected sequentially;
wherein the stacked hourglass module consists of N fourth-stage hourglass modules in series;
each fourth-stage hourglass module comprises five parallel convolutional streams, wherein an innermost convolutional stream is configured to process an original scale, a second to last convolutional stream and an outermost convolutional stream are configured to downsample to ½, ¼, ⅛ and 1/16, respectively; and
the five parallel convolutional streams are configured to extract features in different resolution groups, and deliver the features of each resolution through a residual module, to be recovered to the original scale through an up sample layer and be fused after recovery;
wherein the fourth-stage hourglass module is formed by replacing a residual module at a middle of a fourth row of a third-stage hourglass module with a first-stage hourglass module;
the third-stage hourglass module is formed by replacing a residual module at a middle of a third row of a second-stage hourglass module with the first-stage hourglass module;
the second-stage hourglass module is formed by replacing a residual module at a middle of a second row of the first-stage hourglass module with the first-stage hourglass module; and
the first-stage hourglass module comprises a first row comprising a residual module and a second row comprising a max pool layer, three residual modules and the up sample layer in sequence, wherein the first row and the second row of the first-stage hourglass module are configured to fuse and output the features;
wherein each residual module consists of a first row being a skip level layer comprising the 1×1 convolutional layer, and a second row being a convolutional layer that comprises a batch normalization (BN) layer, a rectified linear unit (Relu) layer, the 1×1 convolutional layer, the BN layer, the Relu layer, the 3×3 convolutional layer, the BN layer, the Relu layer and the 1×1 convolutional layer; and
fusing and outputting the features at outputs of the skip level layer and the convolutional layer.