US 12,293,305 B2
Confidence score based machine learning model training
Sricharan Kallur Palli Kumar, Mountain View, CA (US)
Assigned to Intuit Inc., Mountain View, CA (US)
Filed by Intuit Inc., Mountain View, CA (US)
Filed on May 27, 2021, as Appl. No. 17/331,865.
Prior Publication US 2022/0383152 A1, Dec. 1, 2022
Int. Cl. G06N 5/04 (2023.01); G06N 20/00 (2019.01)
CPC G06N 5/04 (2013.01) [G06N 20/00 (2019.01)] 16 Claims
OG exemplary drawing
 
1. A computer-implemented method for training a machine learning model for making predictions, comprising:
obtaining a plurality of training samples;
generating, by the machine learning model, a plurality of predictions and a plurality of confidence scores by, for each training sample of the plurality of training samples:
generating a prediction by the machine learning model based on the training sample; and
generating, by the machine learning model, a confidence score associated with the prediction by the machine learning model;
excluding one or more training samples of the plurality of training samples for subsequent training of the machine learning model based on one or more confidence scores associated with the one or more training samples; and
training the machine learning model based on the plurality of predictions and associated confidence scores, wherein the machine learning model is further trained using one or more non-excluded training samples of the plurality of training samples, and based on a loss function L(X, y, p), wherein:
the plurality of training samples is {X, y}, X comprising a plurality of input data including input data Xi for integer i from 1 to N, wherein an integer N is a number of training samples in the plurality of training samples;
y comprises a plurality of desired predictions based on X, wherein y includes yi for integer i from 1 to N; and
p comprises the plurality of confidence scores associated with {X, y}, wherein pi is a confidence score associated with yi for integer i from 1 to N.