| CPC G06N 3/082 (2013.01) [G06N 3/0495 (2023.01)] | 6 Claims |
|
1. A neural network construction method having average quantization mechanism, comprising:
retrieving a weight combination comprised in each of network layers of a neural network, wherein the weight combination comprises a plurality of floating-point weights;
generating a loss function according to the weight combination of all the network layers and a plurality of target values, wherein the loss function is a first function of the weight combination and the target values;
corresponding to each of the network layers, calculating a Gini coefficient of the weight combination and accumulating the Gini coefficient of each of the network layers as a regularized correction term, wherein the regularized correction term is a second function of the weight combination;
merging the loss function and the regularized correction term to generate a regularized loss function to perform training on the neural network according to the regularized loss function to keep modifying the floating-point weights in the weight combination, so as to generate a trained weight combination of each of the network layers;
performing quantization on the trained weight combination of each of the network layers to generate a quantized neural network, in which each of the network layers of the quantized neural network comprises the quantization weight combination, wherein the quantization weight combination comprises a plurality of integer weights, and a first distribution of the floating-point weights in the trained weight combination is even such that a second distribution of the integer weights in the quantization weight combination is even; and
implementing the quantized neural network by an embedded system chip.
|