US 12,136,027 B2
Data-dependent node-to-node knowledge sharing by regularization in deep learning
James K. Baker, Maitland, FL (US); and Bradley J. Baker, Berwyn, PA (US)
Assigned to D5AI LLC, Maitland, FL (US)
Filed by D5AI LLC, Maitland, FL (US)
Filed on Jun. 13, 2024, as Appl. No. 18/742,404.
Application 18/742,404 is a continuation of application No. 18/353,698, filed on Jul. 17, 2023, granted, now 12,033,054.
Application 18/353,698 is a continuation of application No. 17/760,398, granted, now 11,741,340, issued on Aug. 29, 2023, previously published as PCT/US2020/027912, filed on Apr. 13, 2020.
Claims priority of provisional application 62/993,163, filed on Mar. 23, 2020.
Prior Publication US 2024/0330644 A1, Oct. 3, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06N 3/04 (2023.01); G06F 18/40 (2023.01)
CPC G06N 3/04 (2013.01) [G06F 18/41 (2023.01)] 31 Claims
OG exemplary drawing
 
1. A computer-implemented method for training a neural network N, wherein:
the neural network N comprises a plurality of layers:
the plurality of layers comprises an input layer, an output layer, and zero or more middle layers that are between the input layer and the output layer; and
each of the plurality of layers comprises one or more nodes, such that the neural network N comprises at least a node P and a node R,
the method comprising:
initial training, by a machine-learning computer system, the neural network N;
receiving, from a human, by a cooperative human-AI learning supervisor computer system of the machine-learning computer system, a confirmation for an interpretation of node P in the initial training; and
in response to receiving the confirmation for the interpretation of node P, resumed training, by the machine-learning computer system, the neural network N to achieve a global objective for the neural network N, wherein the node R has a first knowledge sharing link from the node P to the node R, wherein the resumed training of the neural network N comprises training the neural network N with a set of training data D1 such that the node R is trained with at least two objectives for the set of training data D1, the two objectives comprising:
the global objective for the neural network N; and
a local objective that regularizes node R to improve satisfaction with a specified relationship between, for each datum d in the set of training data D1, (i) an output value of a first mathematical function applied to inputs of node R for the datum d and (ii) an output value of a second mathematical function applied to inputs of the node P for the datum d.