US 11,720,784 B2
Systems and methods for enhancing inferential accuracy of an artificial neural network during training on a mixed-signal integrated circuit
David Fick, Hutto, TX (US); Daniel Graves, Beaumont (CA); Michael Klachko, Boca Raton, FL (US); Ilya Perminov, Vancouver (CA); John Mark Beardslee, Menlo Park, CA (US); and Evgeny Shapiro, Vancouver (CA)
Assigned to Mythic, Inc., Austin, TX (US)
Filed by Mythic, Inc., Austin, TX (US)
Filed on Mar. 25, 2022, as Appl. No. 17/705,250.
Claims priority of provisional application 63/230,538, filed on Aug. 6, 2021.
Claims priority of provisional application 63/175,171, filed on Apr. 15, 2021.
Claims priority of provisional application 63/169,869, filed on Apr. 1, 2021.
Prior Publication US 2022/0318609 A1, Oct. 6, 2022
Int. Cl. G06N 3/06 (2006.01); G06N 3/063 (2023.01); G06N 3/08 (2023.01)
CPC G06N 3/063 (2013.01) [G06N 3/08 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A method for enhancing inferential accuracy of an artificial neural network during training on a mixed-signal integrated circuit, the method comprising:
identifying a dynamic composite scaling factor for each of a plurality of distinct computational layers of a target artificial neural network during a pre-training stage, wherein identifying the dynamic composite scaling factor includes:
(i) executing a pre-training matrix multiply accumulate (MMA) on an analog accelerator for a target computational layer of the plurality of distinct computational layers based on an input of a batch of training data;
(ii) measuring an N-bit activation output signal range of each of a plurality of distinct input channels of the target computational layer of the target artificial neural network based on the execution of the pre-training MMA;
(iii) identifying an input channel equalization factor for each of the plurality of distinct input channels of the target computational layer based on the N-bit activation output signal range of each of the plurality of distinct input channels, wherein the input channel equalization factor, when applied during training, increases an unsealed N-bit activation output signal range of a given input channel to a target N-bit activation output signal range;
(iv) measuring an unsealed N bit activation output signal range of the target computational layer based on the input of the batch of training data;
(v) calculating a dynamic composite scaling factor for the target computational layer based on the unsealed N-bit activation output signal range of the target computation layer; and
(vi) selecting a distinct set of scaling factors for the target computation layer based on the calculated dynamic composite scaling factor for the target computational layer;
during a training of the target artificial neural network:
(a) applying the batch of training data as training input; and
(b) individually applying, during an MMA operation on the analog accelerator, the selected distinct set of scaling factors of the calculated dynamic composite scaling factor to each of the plurality of distinct computational layers of the target artificial neural network.