US 12,462,158 B2
Method and system for training a neural network
Warren J. Gross, Côte-St-Luc (CA); Amir Ardakani, Montreal (CA); and Arash Ardakani, Montreal (CA)
Assigned to THE ROYAL INSTITUTION FOR THE ADVANCEMENT OF LEARNING/MCGILL UNIVERSITY, Montreal (CA)
Filed by THE ROYAL INSTITUTION FOR THE ADVANCEMENT OF LEARNING/MCGILL UNIVERSITY, Montréal (CA)
Filed on Feb. 19, 2021, as Appl. No. 17/180,340.
Claims priority of provisional application 62/978,562, filed on Feb. 19, 2020.
Prior Publication US 2021/0256389 A1, Aug. 19, 2021
Int. Cl. G06N 3/084 (2023.01); G06N 3/045 (2023.01); G06N 3/047 (2023.01); G06N 3/048 (2023.01); G06N 3/063 (2023.01)
CPC G06N 3/084 (2013.01) [G06N 3/047 (2023.01); G06N 3/048 (2023.01)] 19 Claims
OG exemplary drawing
 
1. A computer-implemented method of training a neural network, the neural network comprising a plurality of interconnected nodes arranged in a plurality of layers and having a plurality of weights associated therewith, the method comprising:
obtaining, at a computing device, a training dataset from a database;
performing, at the computing device, a forward propagation on the training dataset to produce an output of the neural network; and
performing, at the computing device, a backward propagation on the output of the neural network by:
computing gradients of the neural network exclusively using ternary operations that estimate a polarity and a neutrality of gradients of the neural network using ternary values represented with two bits, the ternary operations performed by a ternary multiplier realized using an AND gate and an XOR gate, and
updating the plurality of weights using the polarity and the neutrality as estimated.