US 11,954,822 B2
Image processing method and device, training method of neural network, image processing method based on combined neural network model, constructing method of combined neural network model, neural network processor, and storage medium
Pablo Navarrete Michelini, Beijing (CN); Wenbin Chen, Beijing (CN); Hanwen Liu, Beijing (CN); and Dan Zhu, Beijing (CN)
Assigned to BOE TECHNOLOGY GROUP CO., LTD., Beijing (CN)
Appl. No. 17/419,350
Filed by BOE TECHNOLOGY GROUP CO., LTD., Beijing (CN)
PCT Filed Oct. 13, 2020, PCT No. PCT/CN2020/120586
§ 371(c)(1), (2) Date Jun. 29, 2021,
PCT Pub. No. WO2021/073493, PCT Pub. Date Apr. 22, 2021.
Claims priority of application No. 201910995755.2 (CN), filed on Oct. 18, 2019.
Prior Publication US 2022/0084166 A1, Mar. 17, 2022
Int. Cl. G06T 3/4046 (2024.01); G06N 3/08 (2023.01); G06T 3/4053 (2024.01); G06T 5/50 (2006.01)
CPC G06T 3/4046 (2013.01) [G06N 3/08 (2013.01); G06T 3/4053 (2013.01); G06T 5/50 (2013.01)] 16 Claims
OG exemplary drawing
 
1. An image processing method, comprising:
obtaining an input image;
obtaining, based on the input image, initial feature images of N stages with resolutions from high to low, wherein N is a positive integer and N>2;
performing, based on initial feature images of second to N-th stages, cyclic scaling processing on an initial feature image of a first stage, to obtain an intermediate feature image; and
performing merging processing on the intermediate feature image to obtain an output image,
wherein the cyclic scaling processing comprises hierarchically-nested scaling processing of N−1 stages, and scaling processing of each stage comprises down-sampling processing, concatenating processing, up-sampling processing, and residual link addition processing;
down-sampling processing of an i-th stage performs, based on an input of scaling processing of the i-th stage, down-sampling to obtain a down-sampling output of the i-th stage,
concatenating processing of the i-th stage performs, based on the down-sampling output of the i-th stage and an initial feature image of an (i+1)-th stage, concatenating to obtain a concatenating output of the i-th stage,
up-sampling processing of the i-th stage obtains an up-sampling output of the i-th stage based on the concatenating output of the i-th stage, and
residual link addition processing of the i-th stage performs residual link addition between the input of the scaling processing of the i-th stage and the up-sampling output of the i-th stage, to obtain an output of the scaling processing of the i-th stage, wherein i=1, 2, . . . , N−1; and
scaling processing of a (j+1)-th stage is nested between down-sampling processing of a j-th stage and concatenating processing of the j-th stage, and an output of the down-sampling processing of the j-th stage serves as an input of the scaling processing of the (j+1)-th stage, wherein j=1, 2, . . . , N−2,
wherein the obtaining, based on the input image, the initial feature images of the N stages with resolutions from high to low, comprises:
concatenating the input image with a random noise image to obtain a concatenating input image; and
performing analysis processing of N different stages on the concatenating input image, to obtain the initial feature images of the N stages with resolutions from high to low, respectively.