US 11,727,268 B2
Sparse training in neural networks
Zai Wang, Pudong New Area (CN); Shengyuan Zhou, Pudong New Area (CN); Shuai Hu, Pudong New Area (CN); and Tianshi Chen, Pudong New Area (CN)
Assigned to SHANGHAI CAMBRICON INFORMATION TECHNOLOGY CO., LTD., Pudong New Area (CN)
Filed by Shanghai Cambricon Information Technology Co., Ltd., Pudong New Area (CN)
Filed on Nov. 28, 2019, as Appl. No. 16/698,979.
Application 16/698,979 is a continuation of application No. 16/698,976, filed on Nov. 28, 2019, granted, now 11,544,542.
Application 16/698,976 is a continuation in part of application No. PCT/CN2018/090901, filed on Jun. 12, 2018.
Claims priority of application No. 201710473955.2 (CN), filed on Jun. 21, 2017.
Prior Publication US 2020/0104713 A1, Apr. 2, 2020
Int. Cl. G06N 3/08 (2023.01); G06N 3/084 (2023.01); G06N 3/047 (2023.01)
CPC G06N 3/08 (2013.01) [G06N 3/047 (2023.01); G06N 3/084 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A sparse training method comprising:
selectively zeroing one or more gradients corresponding to one or more neurons included in a layer of a neural network according to a zero-setting condition;
performing training operations by using one or more non-zeroed gradients to obtain updated gradients and synapses; and
screening the one or more neurons included in the layer randomly prior to zeroing corresponding gradients of selected neurons according to the zero-setting condition,
wherein the zero-setting condition is a zero-setting probability condition, wherein the zero-setting probability is p, N*p neurons are selected randomly, and corresponding gradients of the N*p neurons are set to zero.