US 12,353,974 B2
Data-dependent training for automated knowledge system that comprises a neural network
James K. Baker, Maitland, FL (US); and Bradley J. Baker, Berwyn, PA (US)
Assigned to D5AI LLC, Maitland, FL (US)
Filed by D5AI LLC, Maitland, FL (US)
Filed on Oct. 3, 2024, as Appl. No. 18/905,506.
Application 18/905,506 is a continuation of application No. 18/742,404, filed on Jun. 13, 2024, granted, now 12,136,027.
Application 18/742,404 is a continuation of application No. 18/353,698, filed on Jul. 17, 2023, granted, now 12,033,054, issued on Jul. 9, 2024.
Application 18/353,698 is a continuation of application No. 17/760,398, granted, now 11,741,340, issued on Aug. 29, 2023, previously published as PCT/US2020/027912, filed on Apr. 13, 2020.
Claims priority of provisional application 62/993,163, filed on Mar. 23, 2020.
Prior Publication US 2025/0036914 A1, Jan. 30, 2025
Int. Cl. G06N 3/04 (2023.01); G06F 18/40 (2023.01)
CPC G06N 3/04 (2013.01) [G06F 18/41 (2023.01)] 51 Claims
OG exemplary drawing
 
1. A method of training an automated knowledge system to perform an automated task, wherein the automated knowledge system comprises a neural network, where the neural network comprises l=0, . . . , L layers, where L is greater than or equal to 2, wherein each layer comprises at least one node such that the neural network comprises a node R such that the node R is not on the l=0 layer, wherein the method trains the automated knowledge system to make a behavior of the node R of the neural network more interpretable to humans, the method comprising:
iteratively training at least the neural network of the automated knowledge system, wherein the iterative training comprises, for at least one iteration:
in a forward pass through the neural network, computing, by a programmed computer system, an activation value for the node R for a training datum d;
determining, by the programmed computer system, whether a specified relationship between a first computed value for the node R for the training datum d and a second computed value for a digital knowledge source P for the training datum d is violated, wherein:
the first computed value for the node R for the training datum d is related to the activation value for the node R for the training datum d; and
the second computed value for the digital knowledge source P for the training datum d is related to an activation value for the digital knowledge source P for the training datum d;
upon a determination that the specified relationship is violated, imposing, by the programmed computer system, a penalty term on a cost function for the node R; and
adjusting the automated knowledge system by updating, by the programmed computer system, learned parameters for the node R based on the cost function for the node R with the penalty term imposed; and
after the iterative training, performing, by the automated knowledge system, the automated task.