US 12,079,718 B2
Device, method and program for erasing select contributing layers of a neural network
Yasutoshi Ida, Musashino (JP)
Assigned to NIPPON TELEGRAPH AND TELEPHONE CORPORATION, Tokyo (JP)
Appl. No. 16/980,430
Filed by NIPPON TELEGRAPH AND TELEPHONE CORPORATION, Tokyo (JP)
PCT Filed Apr. 4, 2019, PCT No. PCT/JP2019/015040
§ 371(c)(1), (2) Date Sep. 14, 2020,
PCT Pub. No. WO2019/194299, PCT Pub. Date Oct. 10, 2019.
Claims priority of application No. 2018-073498 (JP), filed on Apr. 5, 2018.
Prior Publication US 2020/0410348 A1, Dec. 31, 2020
Int. Cl. G06N 3/08 (2023.01); G06N 3/04 (2023.01)
CPC G06N 3/08 (2013.01) [G06N 3/04 (2013.01)] 12 Claims
OG exemplary drawing
 
1. A learning device, comprising:
a memory; and
a processor, coupled to the memory, configured to:
calculate, for each layer of a plurality of layers within a multilayer neural network, a degree of contribution based on a norm of a non-linear mapping corresponding to each layer indicating a degree of contribution to an estimation result of the multilayer neural network, wherein the multilayer neural network is a residual network and the plurality of layers include residual units that are stacked to construct the multilayer neural network;
select one or more to-be-erased layers from the plurality of layers within the multilayer neural network based on comparing the degree of contribution of each layer of the multilayer neural network and selecting a predetermined number of layers having lower degrees of contribution than all other layers not selected;
erase the one of more to-be-erased layers from the multilayer neural network by erasing the non-linear mapping from residual units of the multilayer neural network corresponding to the one or more to-be-erased layers;
retrain the multilayer neural network from which the one or more to-be-erased layers have been erased, wherein the learning device performs estimation with decreased memory consumption after the one or more to-be-erased layers have been erased; and
perform image recognition using the retrained multilayer neural network.