US 11,676,026 B2
Using back propagation computation as data
James K. Baker, Maitland, FL (US)
Assigned to D5AI LLC, Maitland, FL (US)
Appl. No. 16/619,325
Filed by D5AI LLC, Maitland, FL (US)
PCT Filed Jun. 4, 2019, PCT No. PCT/US2019/035300
§ 371(c)(1), (2) Date Dec. 4, 2019,
PCT Pub. No. WO2020/005471, PCT Pub. Date Jan. 2, 2020.
Claims priority of provisional application 62/691,907, filed on Jun. 29, 2018.
Prior Publication US 2020/0394521 A1, Dec. 17, 2020
Int. Cl. G06N 3/084 (2023.01); G06N 3/045 (2023.01); G06T 1/20 (2006.01); G06F 18/214 (2023.01)
CPC G06N 3/084 (2013.01) [G06F 18/214 (2023.01); G06N 3/045 (2023.01); G06T 1/20 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method of training a first deep neural network, the method comprising:
[a] training the first deep neural network with a first training data example, wherein training the first deep neural network with a first training data example comprises:
a feedforward computation through the first deep neural network; and
a back-propagation computation, with respect to a first network objective function for the first deep neural network, through the first deep neural network at least to a target node in a hidden layer of the first deep neural network, wherein the first network objective function is for measuring quality of output of the first deep neural network, wherein the back-propagation computation comprises, at least, an estimated partial derivative of the first network objective with respect to an output of the target node, an input to the target node, and/or a connection weight for a connection to the target node from another node in the first deep neural network;
[b] training a second deep neural network with a second network objective function that is different from the first network objective function, wherein the second network objective function is for measuring quality of output of the second deep neural network, wherein training the second deep neural network comprises inputting to an input layer of the second deep neural network at least the estimated partial derivative computed in the back-propagation computation for the first deep neural network relative to the target node, such that the second deep neural network is trained to compute an output based on estimated partial derivatives computed in the back-propagation computation for the first deep neural network relative to the target node; and
[c] improving the first deep neural network based on an output of the second deep neural network.