CPC G06N 3/084 (2013.01) [G06F 18/214 (2023.01); G06N 3/045 (2023.01); G06T 1/20 (2013.01)] | 20 Claims |
1. A method of training a first deep neural network, the method comprising:
[a] training the first deep neural network with a first training data example, wherein training the first deep neural network with a first training data example comprises:
a feedforward computation through the first deep neural network; and
a back-propagation computation, with respect to a first network objective function for the first deep neural network, through the first deep neural network at least to a target node in a hidden layer of the first deep neural network, wherein the first network objective function is for measuring quality of output of the first deep neural network, wherein the back-propagation computation comprises, at least, an estimated partial derivative of the first network objective with respect to an output of the target node, an input to the target node, and/or a connection weight for a connection to the target node from another node in the first deep neural network;
[b] training a second deep neural network with a second network objective function that is different from the first network objective function, wherein the second network objective function is for measuring quality of output of the second deep neural network, wherein training the second deep neural network comprises inputting to an input layer of the second deep neural network at least the estimated partial derivative computed in the back-propagation computation for the first deep neural network relative to the target node, such that the second deep neural network is trained to compute an output based on estimated partial derivatives computed in the back-propagation computation for the first deep neural network relative to the target node; and
[c] improving the first deep neural network based on an output of the second deep neural network.
|