US 11,956,453 B2
Content-adaptive online training for DNN-based cross component prediction with scaling factors
Sheng Lin, San Jose, CA (US); Wei Jiang, Sunnyvale, CA (US); Wei Wang, Palo Alto, CA (US); Ding Ding, Palo Alto, CA (US); Shan Liu, San Jose, CA (US); and Xiaozhong Xu, State College, PA (US)
Assigned to TENCENT AMERICA LLC, Palo Alto, CA (US)
Filed by TENCENT AMERICA LLC, Palo Alto, CA (US)
Filed on May 26, 2022, as Appl. No. 17/825,339.
Claims priority of provisional application 63/210,762, filed on Jun. 15, 2021.
Prior Publication US 2022/0400272 A1, Dec. 15, 2022
Int. Cl. H04N 19/42 (2014.01); H04N 19/147 (2014.01); H04N 19/186 (2014.01); H04N 19/30 (2014.01); H04N 19/50 (2014.01)
CPC H04N 19/42 (2014.11) [H04N 19/147 (2014.11); H04N 19/186 (2014.11); H04N 19/30 (2014.11); H04N 19/50 (2014.11)] 20 Claims
OG exemplary drawing
 
1. A method for neural network (NN) based cross component prediction with scaling factors during encoding or decoding, the method being executed by one or more processors, the method comprising:
training a deep neural network (DNN) cross component prediction (CCP) model with at least one or more scaling factors,
wherein the at least one or more scaling factors of the DNN CCP are learned by optimizing a rate-distortion loss based on an input video sequence comprising a luma component, and
wherein the at least one or more scaling factors of the DNN CCP are learned in addition to one or more bias parameters of the DNN CCP or one or more weight parameters of the DNN CCP;
reconstructing a chroma component based on the luma component using the DNN CCP model with the at least one or more scaling factors for chroma prediction;
updating the DNN CCP model for chroma prediction of the input video sequence using the one or more scaling factors; and
performing chroma prediction of the input video sequence using the updated DNN CCP model with the one or more scaling factors.