US 12,283,075 B2
Method and apparatus for multi-learning rates of substitution in neural image compression
Ding Ding, Palo Alto, CA (US); Wei Jiang, Sunnyvale, CA (US); Sheng Lin, San Jose, CA (US); Wei Wang, Palo Alto, CA (US); Xiaozhong Xu, State College, PA (US); and Shan Liu, San Jose, CA (US)
Assigned to TENCENT AMERICA LLC, Palo Alto, CA (US)
Filed by TENCENT AMERICA LLC, Palo Alto, CA (US)
Filed on Oct. 13, 2021, as Appl. No. 17/499,959.
Claims priority of provisional application 63/176,204, filed on Apr. 16, 2021.
Prior Publication US 2022/0343552 A1, Oct. 27, 2022
Int. Cl. H04N 19/184 (2014.01); G06N 3/08 (2023.01); G06T 9/00 (2006.01); G11B 20/00 (2006.01)
CPC G06T 9/002 (2013.01) [G06N 3/08 (2013.01); H04N 19/184 (2014.11); G06F 2212/455 (2013.01); G11B 20/00007 (2013.01); H04Q 2213/034 (2013.01)] 9 Claims
OG exemplary drawing
 
1. A method of substitutional end-to-end (E2E) neural image compression (NIC) using a neural network performed by at least one processor, the method comprising:
receiving an input image to an E2E NIC framework;
splitting the input image into one or more blocks;
performing an encoding mapping, for each of the one or more blocks, by mapping the input image to a first bitstream having a first length;
performing a decoding mapping, for each of the one or more blocks, by mapping the first bitstream back to an original space with a first distortion loss;
determining a substitute image from the original space, based on a training model of the E2E NIC framework;
encoding the substitute image to generate a second bitstream; and
mapping the substitute image to the second bitstream to generate a compressed representation,
wherein the training model of the E2E NIC framework is trained based on a learning rate of the input image, a quantity of updates to the input image, and a second distortion loss,
wherein a plurality of substitute images are determined based on learning rates that are selected based on characteristics of the input image, and
wherein the substitute image is determined by performing an optimization process of the training model of the E2E NIC framework, comprising:
adjusting RGB variance of the split blocks to generate substitute block representations; and
selecting the RGB variance with a least distortion loss between the split blocks and the substitute block representations to use as the substitute block.