US 12,327,182 B2
Optimizations for analog hardware realization of trained neural networks
Aleksandrs Timofejevs, Riga (LV); Boris Maslov, Newport Beach, CA (US); Nikolai Kovshov, Moscow (RU); and Dmitri Godovskiy, Moscow (RU)
Assigned to PolyN Technology Limited, Bristol (GB)
Filed by PolyN Technology Limited, London (GB)
Filed on Mar. 11, 2021, as Appl. No. 17/199,373.
Application 17/199,373 is a continuation of application No. 17/189,109, filed on Mar. 1, 2021.
Application 17/189,109 is a continuation of application No. PCT/RU2020/000306, filed on Jun. 25, 2020.
Application 17/189,109 is a continuation of application No. PCT/EP2020/067800, filed on Jun. 25, 2020.
Prior Publication US 2021/0406664 A1, Dec. 30, 2021
Int. Cl. G06N 3/065 (2023.01); G06F 1/3206 (2019.01); G06F 1/3287 (2019.01); G06F 30/39 (2020.01); G06N 3/044 (2023.01); G06N 3/049 (2023.01); G06N 3/0499 (2023.01); G06N 3/063 (2023.01); G06N 3/082 (2023.01); G06N 5/04 (2023.01)
CPC G06N 3/065 (2023.01) [G06F 1/3206 (2013.01); G06F 1/3287 (2013.01); G06F 30/39 (2020.01); G06N 3/044 (2023.01); G06N 3/049 (2013.01); G06N 3/0499 (2023.01); G06N 3/063 (2013.01); G06N 3/082 (2013.01); G06N 5/04 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A method for hardware realization of neural networks, comprising:
obtaining a neural network topology and weights of a trained neural network;
transforming the neural network topology to an equivalent analog network of analog components including a plurality of operational amplifiers and a plurality of resistors, wherein each operational amplifier represents an analog neuron of the equivalent analog network and each resistor represents a connection between two analog neurons;
computing a weight matrix for the equivalent analog network based on the weights of the trained neural network, wherein each element of the weight matrix represents a respective connection;
generating a resistance matrix for the weight matrix, wherein each element of the resistance matrix corresponds to a respective weight of the weight matrix;
pruning the equivalent analog network to reduce number of the plurality of operational amplifiers or the plurality of resistors, based on the resistance matrix, to obtain an optimized analog network of analog components; and
generating a schematic model for implementing the optimized analog network as electronic circuitry based on the resistance matrix, including:
selecting component values for each individual operational amplifier and each individual resistor, and
for each analog neuron of the equivalent analog network:
computing a respective bias value for the respective analog neuron based on the weights of the trained neural network, while computing the weight matrix;
in accordance with a determination that the respective bias value is above a predetermined maximum bias threshold, removing the respective analog neuron from the equivalent analog network; and
in accordance with a determination that the respective bias value is below a predetermined minimum bias threshold, replacing the respective analog neuron with a linear junction in the equivalent analog network.