US 12,443,626 B2
Method and system for evaluation of classification models
Danial Dervovic, London (GB); Michael Cashmore, Stirlingshire (GB); and Daniele Magazzeni, London (GB)
Assigned to JPMORGAN CHASE BANK, N.A., New York, NY (US)
Filed by JPMorgan Chase Bank, N.A., New York, NY (US)
Filed on Oct. 10, 2023, as Appl. No. 18/378,484.
Prior Publication US 2025/0117404 A1, Apr. 10, 2025
Int. Cl. G06F 16/00 (2019.01); G06F 11/34 (2006.01); G06F 16/28 (2019.01)
CPC G06F 16/285 (2019.01) [G06F 11/3452 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A method for evaluating a classification model, the method being implemented by at least one processor, the method comprising:
receiving, by the at least one processor, first information that relates to first data to be used for training and evaluating a performance of a first classification model that is designed to make a determination with respect to a predetermined query;
analyzing, by the at least one processor, the first information to determine second information that relates to a known portion of the first information and third information that relates to missing data;
estimating, by the at least one processor based on the third information, an uncertainty that corresponds to the missing data; and
calculating, by the at least one processor based on the second information and the estimated uncertainty, a first Gaussian approximation to a performance metric that relates to the first classification model,
wherein the first data is an incomplete data set for which data is known to be missing, and
wherein the predetermined query relates to at least one from among a financial fraud detection query, an infectious disease classification query, and an e-commerce data stream query, and
wherein the calculating of the first Gaussian approximation to the performance metric comprises estimating a mean value of the performance metric and estimating a variance value of the performance metric.