US 12,443,843 B2
Machine learning uncertainty quantification and modification
Scott Michael Zoldi, San Diego, CA (US); Jeremy Mamer Schmitt, Encinitas, CA (US); and Maria Edna Derderian, San Diego, CA (US)
Assigned to Fair Isaac Corporation, Minneapolis, MN (US)
Filed by FAIR ISAAC CORPORATION, Roseville, MN (US)
Filed on Sep. 13, 2021, as Appl. No. 17/473,250.
Prior Publication US 2023/0080851 A1, Mar. 16, 2023
Int. Cl. G06N 3/08 (2023.01); G06F 17/18 (2006.01); G06N 3/045 (2023.01); G06N 20/20 (2019.01)
CPC G06N 3/08 (2013.01) [G06F 17/18 (2013.01); G06N 3/045 (2023.01); G06N 20/20 (2019.01)] 16 Claims
OG exemplary drawing
 
1. A computer-implemented method, the method comprising:
determining, by one or more programmable processors, an uncertainty value associated with a first machine learning model output of a first machine learning model;
generating, by the one or more programmable processors, a confidence interval for the first machine learning model output associated with an input;
switching, by the one or more programmable processors and responsive to the uncertainty value satisfying a threshold, from the first machine learning model to a second machine learning model, the second machine learning model generating a second machine learning model output;
generating, by the one or more programmable processors, the second machine learning model based on the first machine learning model; and
providing, by the one or more programmable processors and responsive to the switching, the first machine learning output, the uncertainty value, the confidence interval, and the second machine learning output to a user interface,
wherein generating the second machine learning model based on the first machine learning model comprises:
constructing hidden layers of the second machine learning model where hidden nodes of the hidden layers are a sparse sub-network of hidden nodes approximating the first machine learning model;
generating perturbed variations of sparse networks of high variance hidden nodes;
removing or prohibiting feature interactions contributing the high variance hidden nodes; and
iterating and training the second machine learning model based on removed and prohibited feature interactions to minimize model variance of the second machine learning model.