US 12,248,871 B2
Methods and systems for training multi-bit spiking neural networks for efficient implementation on digital hardware
Aaron Russell Voelker, Stittsville (CA)
Assigned to APPLIED BRAIN RESEARCH INC., Waterloo (CA)
Filed by Applied Brain Research Inc., Waterloo (CA)
Filed on Oct. 30, 2020, as Appl. No. 17/086,218.
Claims priority of provisional application 62/964,266, filed on Jan. 22, 2020.
Claims priority of provisional application 62/929,722, filed on Nov. 1, 2019.
Prior Publication US 2021/0133568 A1, May 6, 2021
Int. Cl. G06N 3/08 (2023.01); G06F 7/58 (2006.01); G06N 3/048 (2023.01); G06N 3/063 (2023.01)
CPC G06N 3/08 (2013.01) [G06F 7/588 (2013.01); G06N 3/048 (2023.01); G06N 3/063 (2013.01)] 23 Claims
OG exemplary drawing
 
1. A method for producing neural network output for a machine learning task, the method implemented with a processor in digital hardware comprising at least one microcontroller or integrated circuit, the method comprising:
providing a non-spiking neural network comprising one or more processing nodes that each implement a nonlinear function;
augmenting, using temporal dithering, by the at least one microcontroller or integrated circuit, at least one nonlinear function of the neural network to provide at least one augmented neural nonlinearity, the augmenting comprising:
evaluating, by the at least one microcontroller or integrated circuit, an output of the at least one nonlinear function;
multiplying, by the at least one microcontroller or integrated circuit, the output of the at least one nonlinear function with a time-step parameter to obtain a first result corresponding to a number of spikes;
producing, by the at least one microcontroller or integrated circuit, a multi-bit spike output for the at least one augmented neural nonlinearity, the producing comprising dividing a second result by the time-step parameter, the second result being obtained from the first result;
training, by the at least one microcontroller or integrated circuit, the neural network with the at least one augmented neural nonlinearity while adjusting the time-step parameter to reduce quantization error, to provide a trained neural network with at least one augmented neural nonlinearity; and
operating, by the at least one microcontroller or integrated circuit, the trained neural network with the at least one augmented neural nonlinearity to produce a temporally sparse sequence of multi-bit spikes in response to input signals.