US 11,876,988 B2
Method and apparatus for task-adaptive pre-processing for neural image compression
Wei Jiang, Sunnyvale, CA (US); Wei Wang, San Jose, CA (US); Ding Ding, Palo Alto, CA (US); Shan Liu, San Jose, CA (US); and Xiaozhong Xu, State College, PA (US)
Assigned to TENCENT AMERICA LLC, Palo Alto, CA (US)
Filed by TENCENT AMERICA LLC, Palo Alto, CA (US)
Filed on Jul. 1, 2021, as Appl. No. 17/365,395.
Claims priority of provisional application 63/138,901, filed on Jan. 19, 2021.
Prior Publication US 2022/0232232 A1, Jul. 21, 2022
Int. Cl. H04N 19/42 (2014.01); H04N 19/147 (2014.01); G06F 18/214 (2023.01); G06V 10/82 (2022.01)
CPC H04N 19/42 (2014.11) [G06F 18/214 (2023.01); G06V 10/82 (2022.01); H04N 19/147 (2014.11)] 20 Claims
OG exemplary drawing
 
1. A method of task-adaptive pre-processing (TAPP) for neural image compression, the method being performed by at least one processor, and the method comprising:
generating a substitutional image, based on an input image, using a TAPP neural network; and
encoding the generated substitutional image to generate a compressed representation, using a first neural network,
wherein the TAPP neural network is trained by:
generating a substitutional training image, based on an input training image, using the TAPP neural network;
encoding the generated substitutional training image to generate a compressed training representation, using the first neural network;
decoding the generated compressed training representation to reconstruct an output training image, using a second neural network;
generating gradients of a rate-distortion (R-D) loss that is generated based on the input training image, the reconstructed output training image and the generated compressed training representation; and
updating the generated substitutional training image, based on the generated gradients of the R-D loss.