US 12,242,969 B2
Graph diffusion for structured pruning of neural networks
Honglei Zhang, Tampere (FI); Francesco Cricri, Tampere (FI); Hamed Rezazadegan Tavakoli, Espoo (FI); Joachim Wabnig, Cambourne (GB); Iraj Saniee, New Providence, NJ (US); Miska Matias Hannuksela, Tampere (FI); and Emre Aksu, Tampere (FI)
Assigned to Nokia Technologies Oy, Espoo (FI)
Filed by Nokia Technologies Oy, Espoo (FI)
Filed on Jun. 22, 2021, as Appl. No. 17/354,398.
Claims priority of provisional application 63/042,186, filed on Jun. 22, 2020.
Prior Publication US 2021/0397965 A1, Dec. 23, 2021
Int. Cl. G06N 3/08 (2023.01); G06N 3/04 (2023.01); G06N 3/082 (2023.01)
CPC G06N 3/082 (2013.01) [G06N 3/04 (2013.01)] 26 Claims
OG exemplary drawing
 
1. An apparatus comprising:
at least one processor; and
at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to:
estimate an importance of parameters of a neural network based on a graph diffusion process over at least one layer of the neural network;
determine the parameters of the neural network for pruning and sparsification, based on the importance of the parameters of the neural network estimated based on the graph diffusion process;
remove neurons of the neural network to prune the neural network, based on the parameters of the neural network determined for pruning;
apply data dependent-based sparsification of the neural network with regard to a sparsification ratio to set a value of at least one weight of the neural network to zero; and
provide, to a receiver, at least one syntax element for signaling the pruned and sparsified neural network, wherein the at least one syntax element comprises at least one neural network representation syntax element;
wherein the at least one syntax element comprises a sparse flag to indicate sparsification is applied.