US 11,995,554 B2
Apparatus and methods for backward propagation in neural networks supporting discrete data
Qi Guo, Beijing (CN); Yong Yu, Beijing (CN); Tianshi Chen, Beijing (CN); and Yunji Chen, Beijing (CN)
Assigned to CAMBRICON TECHNOLOGIES CORPORATION LIMITED, Beijing (CN)
Filed by Cambricon Technologies Corporation Limited, Beijing (CN)
Filed on Nov. 6, 2018, as Appl. No. 16/182,439.
Application 16/182,439 is a continuation in part of application No. 16/093,958, abandoned, previously published as PCT/CN2016/079443, filed on Apr. 15, 2016.
Prior Publication US 2019/0080241 A1, Mar. 14, 2019
Int. Cl. G06N 3/084 (2023.01); G06N 3/063 (2023.01); G06F 9/30 (2018.01); G06F 9/38 (2018.01); G06N 3/04 (2023.01); G06N 3/048 (2023.01); G06N 20/00 (2019.01)
CPC G06N 3/084 (2013.01) [G06N 3/063 (2013.01); G06F 9/3016 (2013.01); G06F 9/3826 (2013.01); G06N 3/04 (2013.01); G06N 3/048 (2023.01); G06N 20/00 (2019.01)] 23 Claims
OG exemplary drawing
 
1. An apparatus for backpropagation of a multilayer neural network (MNN), comprising:
a computation circuit configured to receive one or more groups of MNN data,
wherein the one or more groups of MNN data include input data and one or more weight values,
wherein at least a portion of the input data and the weight values are presented as discrete values,
wherein each of the discrete values is associated with a predetermined bitwise operation to be performed on the one or more groups of MNN data,
wherein an association between each of the discrete values and the predetermined bitwise operation is stored in a lookup table,
wherein the lookup table corresponds to one of one or more arithmetic operations, and
wherein the computation circuit includes:
a master computation circuit configured to calculate an input gradient vector based on a first output gradient vector from an adjacent layer and based on a data type of each of the one or more groups of MNN data, wherein the master computation circuit is further configured to select, from multiple predetermined bitwise operations, one or more operations corresponding to the discrete values and perform the one or more selected operations to calculate the input gradient vector, and
one or more slave computation circuits configured to parallelly calculate portions of a second output vector based on the input gradient vector calculated by the master computation circuit and based on the data type of each of the one or more groups of MNN data; and
a controller circuit configured to decode an instruction that initiates a backpropagation process and transmit the decoded instruction to the computation circuit.