US 11,900,640 B2
Method and apparatus for substitutional neural residual compression
Wei Jiang, San Jose, CA (US); Wei Wang, Palo Alto, CA (US); and Shan Liu, San Jose, CA (US)
Assigned to TENCENT AMERICA LLC, Palo Alto, CA (US)
Filed by TENCENT AMERICA LLC, Palo Alto, CA (US)
Filed on Apr. 21, 2021, as Appl. No. 17/236,108.
Claims priority of provisional application 63/052,242, filed on Jul. 15, 2020.
Prior Publication US 2022/0020182 A1, Jan. 20, 2022
Int. Cl. H04N 19/172 (2014.01); G06T 9/00 (2006.01); G06T 3/40 (2006.01); H04N 19/137 (2014.01); G06N 3/08 (2023.01); H04N 19/149 (2014.01); G06N 3/045 (2023.01)
CPC G06T 9/002 (2013.01) [G06N 3/045 (2023.01); G06N 3/08 (2013.01); G06T 3/4046 (2013.01); H04N 19/137 (2014.11); H04N 19/149 (2014.11); H04N 19/172 (2014.11)] 20 Claims
OG exemplary drawing
 
1. A method of substitutional neural residual compression, the method being performed by at least one processor, and the method comprising:
estimating motion vectors, based on a current image frame and a previous reconstructed image frame;
obtaining a predicted image frame, based on the estimated motion vectors and the previous reconstructed image frame;
subtracting the obtained predicted image frame from the current image frame to obtain a substitutional residual;
encoding the obtained substitutional residual, using a first neural network, to obtain an encoded representation; and
compressing the encoded representation,
wherein the first neural network is trained by updating the obtained substitutional residual to reduce a determined gradient of rate-distortion loss.