US 12,192,440 B2
Neural network based residual coding and prediction for predictive coding
Jiefu Zhai, San Jose, CA (US); Xingyu Zhang, Cupertino, CA (US); Xiaosong Zhou, Campbell, CA (US); Jun Xin, Sunnyvale, CA (US); Hsi-Jung Wu, San Jose, CA (US); and Yeping Su, Sunnyvale, CA (US)
Assigned to APPLE INC., Cupertino, CA (US)
Filed by Apple Inc., Cupertino, CA (US)
Filed on Jan. 4, 2022, as Appl. No. 17/568,266.
Application 17/568,266 is a continuation of application No. 16/254,528, filed on Jan. 22, 2019, granted, now 11,240,492.
Prior Publication US 2022/0191473 A1, Jun. 16, 2022
Int. Cl. H04N 19/105 (2014.01); G06N 3/08 (2023.01); H04N 19/147 (2014.01); H04N 19/159 (2014.01); H04N 19/176 (2014.01); H04N 19/61 (2014.01)
CPC H04N 19/105 (2014.11) [G06N 3/08 (2013.01); H04N 19/147 (2014.11); H04N 19/159 (2014.11); H04N 19/176 (2014.11); H04N 19/61 (2014.11)] 22 Claims
OG exemplary drawing
 
1. A method for coding a video stream, comprising:
for a pixel block of an input frame to be coded, generating a pixel block prediction based on input data derived from reference data of previously-coded data of the video stream;
generating a residual block representing a difference between the pixel block and the pixel block prediction;
coding the residual block; and
packing the coded residual block and associated coding parameters in a coded video stream;
wherein at least one of the generating a pixel block prediction and the coding the residual block is performed using a plurality of neural networks banks, and
wherein different ones of the plurality of neural network banks are trained to operate at a different bitrates from each other.