US 12,136,186 B2
Super resolution image processing method and apparatus
Long Sun, Shenzhen (CN); Baiqi He, Shenzhen (CN); and Meng Chen, Hangzhou (CN)
Assigned to Huawei Technologies Co., Ltd., Shenzhen (CN)
Filed by HUAWEI TECHNOLOGIES CO., LTD., Guangdong (CN)
Filed on Mar. 4, 2022, as Appl. No. 17/687,298.
Application 17/687,298 is a continuation of application No. PCT/CN2020/108829, filed on Aug. 13, 2020.
Claims priority of application No. 201910840356.9 (CN), filed on Sep. 6, 2019.
Prior Publication US 2022/0188976 A1, Jun. 16, 2022
Int. Cl. G06T 3/4053 (2024.01); H04N 19/176 (2014.01); H04N 19/513 (2014.01); H04N 19/85 (2014.01)
CPC G06T 3/4053 (2013.01) [H04N 19/176 (2014.11); H04N 19/513 (2014.11); H04N 19/85 (2014.11)] 18 Claims
OG exemplary drawing
 
1. An image processing method, wherein the method comprises:
obtaining a video stream, wherein the video stream comprises a first frame of image, a second frame of image, and a third frame of image that are adjacent in time sequence;
decoding the video stream to obtain a first alignment frame, a second alignment frame, and at least one residual between the first frame of image, the second frame of image, and the third frame of image, wherein the first alignment frame is generated after the first frame of image moves a pixel block towards the second frame of image based on a first motion vector, the second alignment frame is generated after the third frame of image moves a pixel block towards the second frame of image based on a second motion vector, and each residual of the at least one residual is a pixel difference between a first macroblock in a previous frame of image and a second macroblock corresponding to the first macroblock in a subsequent frame of image after the previous frame of image performs motion compensation towards the subsequent frame of image based on a motion vector;
generating at least one residual frame based on the at least one residual; and
performing super resolution processing on the second frame of image based on the at least one residual frame, the first alignment frame, and the second alignment frame to obtain an additional second frame of image obtained after super resolution, wherein performing the super resolution processing comprises:
generating high-frequency information based on the at least one residual frame;
generating a luminance channel based on the first alignment frame, the second alignment frame, and the second frame of image; and
merging the high-frequency information with the luminance channel to generate the additional second frame of image.