US 12,469,112 B2
Image restoration method and apparatus, and electronic device
Ran Duan, Beijing (CN); and Dan Zhu, Beijing (CN)
Assigned to Beijing BOE Technology Development Co., Ltd., Beijing (CN)
Appl. No. 17/922,150
Filed by BOE Technology Group Co., Ltd., Beijing (CN)
PCT Filed May 25, 2021, PCT No. PCT/CN2021/095778
§ 371(c)(1), (2) Date Oct. 28, 2022,
PCT Pub. No. WO2021/258959, PCT Pub. Date Dec. 30, 2021.
Claims priority of application No. 202010574404.7 (CN), filed on Jun. 22, 2020.
Prior Publication US 2023/0177652 A1, Jun. 8, 2023
Int. Cl. G06T 5/70 (2024.01); G06T 3/4046 (2024.01); H04N 19/42 (2014.01); H04N 19/86 (2014.01)
CPC G06T 5/70 (2024.01) [G06T 3/4046 (2013.01); H04N 19/42 (2014.11); H04N 19/86 (2014.11); G06T 2207/10016 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 12 Claims
OG exemplary drawing
 
1. An image restoration method, comprising:
inputting an image to be processed into a target denoising network, wherein the target denoising network comprises a single-frame network and a recursive network, and the image to be processed is any frame in a video to be processed;
removing, via the single-frame network, compression noise of the image to be processed to output a first image;
removing, according to a content of a previous frame image, compression noise of the image to be processed via the recursive network to output a second image, wherein the previous frame image is one previous frame of the image to be processed in the video to be processed; and
performing weighted summation on the first image and the second image, and outputting a denoised image for the image to be processed;
wherein the removing, according to the content of the previous frame image, the compression noise of the image to be processed via the recursive network to output the second image, comprises: removing the compression noise of the image to be processed via at least one first convolution layer, at least one first feature series layer and at least one first sampling layer cascaded in the recursive network, to output the second image;
wherein the at least one first convolution layer in the recursive network comprises a first sub-convolution layer and a second sub-convolution layer, the at least one first feature series layer comprises a first sub-feature series layer and a second sub-feature series layer, and the at least one first sampling layer comprises first down-sampling layers and first up-sampling layers;
wherein the removing the compression noise of the image to be processed via the at least one first convolution layer, the at least one first feature series layer and the at least one first sampling layer cascaded in the recursive network, to output the second image comprises:
receiving, via the first sub-feature series layer, a first feature image of the image to be processed extracted by each of third sub-convolution layers in second convolution layers in the single-frame network;
obtaining, via the first sub-feature series layer, a second feature image extracted from the previous frame image by the first sub-convolution layer corresponding to each of the third sub-convolution layers in the recursive network;
obtaining a series feature image by performing, via the first sub-feature series layer, series operation on the first feature image and the second feature image;
obtaining a compressed feature image by performing, via each of first sub-convolution layers, compression on the series feature image, wherein the compressed feature images are second feature images extracted from the image to be processed by the first sub-convolution layers;
extracting, via the first down-sampling layers in the at least one first sampling layer, feature images with a plurality of spatial sizes from the compressed feature images;
determining, via the first up-sampling layers, feature images with same spatial sizes as the plurality of spatial sizes;
obtaining a first splicing feature image by splicing, via the second sub-feature series layer, the feature images with the same spatial sizes on feature dimension; and
processing, via the second sub-convolution layer, the first splicing feature image, and outputting the second image.