US 11,869,221 B2
Data compression using integer neural networks
Nicholas Johnston, San Jose, CA (US); and Johannes Balle, San Francisco, CA (US)
Assigned to Google LLC, Mountain View, CA (US)
Appl. No. 17/274,596
Filed by Google LLC, Mountain View, CA (US)
PCT Filed Sep. 18, 2019, PCT No. PCT/US2019/051624
§ 371(c)(1), (2) Date Mar. 9, 2021,
PCT Pub. No. WO2020/068498, PCT Pub. Date Apr. 2, 2020.
Claims priority of provisional application 62/737,852, filed on Sep. 27, 2018.
Prior Publication US 2021/0358180 A1, Nov. 18, 2021
Int. Cl. G06T 9/00 (2006.01); G06F 17/18 (2006.01); G06N 3/084 (2023.01); G06N 3/045 (2023.01)
CPC G06T 9/002 (2013.01) [G06F 17/18 (2013.01); G06N 3/045 (2023.01); G06N 3/084 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method performed by one or more data processing apparatus for entropy encoding data which defines a sequence comprising a plurality of components, wherein each component specifies a respective code symbol from a predetermined discrete set of possible code symbols, the method comprising:
for each component of the plurality of components:
processing an input comprising: (i) a respective integer representation of each of one or more components of the data which precede the component in the sequence, (ii) an integer representation of one or more respective latent variables characterizing the data, or (iii) both, using an integer neural network to generate data defining a probability distribution over the predetermined set of possible code symbols for the component of the data, wherein:
the integer neural network has a plurality of integer neural network parameter values, and each of the plurality of integer neural network parameter values are integers;
the integer neural network comprises a plurality of integer neural network layers, each integer neural network layer is configured to process a respective integer neural network layer input to generate a respective integer neural network layer output, and processing an integer neural network layer input to generate an integer neural network layer output comprises:
generating an intermediate result by processing the integer neural network layer input in accordance with a plurality of integer neural network layer parameters using integer-valued operations; and
generating the integer neural network layer output by applying an integer-valued activation function to the intermediate result; and
generating an entropy encoded representation of the data using the respective probability distribution determined for each component of the plurality of components.