CPC G06N 3/045 (2023.01) | 26 Claims |
1. A method of improving a base neural network, the method comprising:
initial training, at least partially, with a computer system that comprises one or more processing cores, a base neural network through machine learning with a training data set, wherein:
the base neural network comprises an input layer, an output layer, and one or more hidden layers between the input layer and the output layer;
each layer comprises one or more base network nodes, such that the base neural network comprises multiple base network nodes; and
the initial training comprises storing, in a memory of the computer system, preliminary activations values computed for the base network nodes on each of the one or more hidden layers for data in the training data set;
after the initial training of the base neural network, merging, by the computer system, a new node set into the base neural network to form an expanded neural network, wherein:
the new node set comprises one or more nodes; and
merging the new node set and the base neural network comprises directly connecting each of the one or more nodes of the new node set to one or more base network nodes in the expanded neural network; and
after merging the new node set into the base neural network to form the expanded neural network, training, by the computer system, the expanded neural network through iterative machine learning on the training data set using a network error loss function for the expanded neural network, wherein training the expanded neural network comprises imposing a node-to-node relationship regularization for at least one base network node in the expanded neural network, wherein imposing the node-to-node relationship regularization comprises adding, during back-propagation of partial derivatives through the expanded neural network for a datum in the training data set, a regularization cost to the network error loss function for the at least one base network node based on a specified relationship between a stored preliminary activation value for the base network node for the datum and an activation value for the base network node of the expanded neural network for the datum, wherein the preliminary activation value for the base network node of the base neural network for the datum was stored in the memory.
|