US 12,271,812 B2
Neural networks having reduced number of parameters
Attilio Fiandrotti, Turin (IT); Gianluca Francini, Turin (IT); Skjalg Lepsoy, Turin (IT); and Enzo Tartaglione, Turin (IT)
Assigned to TELECOM ITALIA S.p.A., Milan (IT)
Appl. No. 17/251,508
Filed by TELECOM ITALIA S.p.A., Milan (IT)
PCT Filed Jul. 18, 2019, PCT No. PCT/EP2019/069326
§ 371(c)(1), (2) Date Dec. 11, 2020,
PCT Pub. No. WO2020/016337, PCT Pub. Date Jan. 23, 2020.
Claims priority of application No. 102018000007377 (IT), filed on Jul. 20, 2018.
Prior Publication US 2021/0142175 A1, May 13, 2021
Int. Cl. G06N 3/08 (2023.01); G05D 1/00 (2024.01); G06N 3/04 (2023.01)
CPC G06N 3/08 (2013.01) [G05D 1/0088 (2013.01); G06N 3/04 (2013.01)] 10 Claims
OG exemplary drawing
 
1. A method, comprising:
providing a neural network having a set of weights and being configured to receive an input data structure for generating a corresponding output array according to values of said set of weights;
training the neural network to obtain a trained neural network, said training comprising setting values of the set of weights by means of a gradient descent algorithm which exploits a cost function comprising a loss term and a regularization term;
deploying the trained neural network on a device through a communication network; and
using the deployed trained neural network on the device,
wherein the regularization term is based on a rate of change of elements of the output array caused by variations of the set of weights values,
wherein said training further comprises setting to zero weights of the set of weights having a value lower than a corresponding threshold after setting values of the set of weights by means of the gradient descent algorithm,
wherein said regularization term is based on a sum of penalties each one penalizing a corresponding weight of the set of weights, each penalty being based on the product of a first factor and a second factor calculated prior to calculating the sum of the penalties, wherein:
said first factor is based on a power of said corresponding weight, particularly a square of the corresponding weight, and
said second factor is based on a sensitivity function of how sensitive the output array is to a change in the corresponding weight.