US 12,236,201 B1
Enhanced machine learning model accuracy through post-hoc confidence score calibration
Andrzej Szwabe, Poznan (PL)
Assigned to Snowflake Inc., Bozeman, MT (US)
Filed by Snowflake Inc., Bozeman, MT (US)
Filed on May 29, 2024, as Appl. No. 18/677,561.
Application 18/677,561 is a continuation of application No. 18/450,700, filed on Aug. 16, 2023, granted, now 12,032,919.
Int. Cl. G06F 40/40 (2020.01); G06F 40/284 (2020.01)
CPC G06F 40/40 (2020.01) [G06F 40/284 (2020.01)] 30 Claims
OG exemplary drawing
 
1. A method comprising:
receiving, by at least one hardware processor, results generated by a machine learning (ML) model, the results comprising at least one confidence score and a plurality of electronic documents;
processing the results generated by the ML model, the processing comprising performing document understanding by extracting data points from the plurality of electronic documents;
associating the at least one confidence score with the extracted data points;
calibrating the at least one confidence score associated with the extracted data points using a post-hoc calibration (PHC) solution set;
implementing confidence scoring recalibration, the confidence scoring recalibration comprising functionality to assess reliability of the results generated by the ML model, the confidence scoring recalibration comprising aligning the at least one confidence score with prediction accuracy;
adjusting the at least one confidence score generated by the confidence scoring recalibration;
based on the adjusting of the at least one confidence score, extracting an individual element of information from the plurality of electronic documents, the extracting of the individual element of the information comprising one or more extracted values; and
generating an output comprising the one or more extracted values in a database comprising an adjusted confidence score associated with each of the one or more extracted values and the results generated by a ML model.