US 12,288,317 B2
Method and system for enhancing image quality by multi-frame blending
Gaurav Khandelwal, Bengaluru (IN); Sachin Deepak Lomte, Bengaluru (IN); Umang Chaturvedi, Bengaluru (IN); and Diplav, Bengaluru (IN)
Assigned to SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed by SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed on Aug. 18, 2022, as Appl. No. 17/890,868.
Application 17/890,868 is a continuation of application No. PCT/KR2022/007744, filed on May 31, 2022.
Claims priority of application No. 202141030525 (IN), filed on Jul. 7, 2021; and application No. 202141030525 (IN), filed on Jan. 18, 2022.
Prior Publication US 2023/0014050 A1, Jan. 19, 2023
Int. Cl. G06T 7/00 (2017.01); G06T 5/50 (2006.01); G06T 5/70 (2024.01); G06V 10/44 (2022.01)
CPC G06T 7/0002 (2013.01) [G06T 5/50 (2013.01); G06T 5/70 (2024.01); G06V 10/44 (2022.01); G06T 2207/10016 (2013.01); G06T 2207/20221 (2013.01); G06T 2207/30168 (2013.01)] 10 Claims
OG exemplary drawing
 
1. A method of enhancing image quality, the method comprising:
receiving a plurality of input frames and metadata from an image sensor;
determining a respective plurality of feature scores associated with each input frame of the plurality of input frames;
determining a respective parametric score associated with each input frame based on the respective plurality of feature scores associated with each input frame and the metadata;
identifying one or more artifacts for correction in each input frame based on the respective parametric score;
determining a first type of artifact for which a plurality of first input frames need to be corrected and a second type of artifact for which a plurality of second input frames need to be corrected, with the first type of artifact being different than the second type of artifact;
applying a first transformation associated with the first type of artifact to the plurality of first input frames and a second transformation associated with the second type of artifact to the plurality of second input frames;
performing multi-frame blending for the plurality of first input frames and the plurality of second input frames to which respective transformations have been applied;
estimating a quality of at least one feature of one or more features of a first input frame based on the respective parametric score, wherein the quality is estimated based on at least one of peak signal-to-noise ratio (PSNR), Structural Similarity Index (SSIM) or multi-scale structural similarity (MS-SSIM) rating perceptual quality;
prioritizing the one or more features based on the estimated quality of the first input frame;
determining a strength of correction, based on the estimated quality and the prioritizing, associated with the first type of artifact in the first input frame; and
applying the determined strength of correction in the first input frame based on the respective parametric score,
wherein the determining the respective parametric score comprises:
generating a vector score of at least one of the respective plurality of feature scores of the first input frame based on a weighted average of the respective plurality of feature scores of the first input frame;
blending generated vector score of the respective plurality of feature scores of the first input frame;
scaling the blended vector score; and
correlating one or more vector scores respective of each of the one or more generated feature vectors based on sealed score of the blended vector score.