US 12,248,858 B2
Systems and methods for intelligent generation and assessment of candidate less discriminatory alternative machine learning models
John Wickens-Lamb Merrill, Seattle, WA (US); Kareem Saleh, Marina Del Rey, CA (US); and Mark Eberstein, Marina del Rey, CA (US)
Assigned to Fairness-as-a-Service, Inc., Marina Del Rey, CA (US)
Filed by Fairness-as-a-Service, Inc., Marina Del Rey, CA (US)
Filed on Jun. 12, 2024, as Appl. No. 18/741,000.
Claims priority of provisional application 63/648,890, filed on May 17, 2024.
Claims priority of provisional application 63/537,941, filed on Sep. 12, 2023.
Claims priority of provisional application 63/521,971, filed on Jun. 20, 2023.
Prior Publication US 2024/0428133 A1, Dec. 26, 2024
Int. Cl. G06N 20/00 (2019.01)
CPC G06N 20/00 (2019.01) 18 Claims
OG exemplary drawing
 
1. A computer-implemented method for accelerating a replacement of an automated decisioning model with a disparity-mitigating alternative, the method comprising:
obtaining, via one or more computers, an incumbent automated decisioning model and a candidate disparity-mitigating alternative model for the incumbent automated decisioning model;
generating, via a resampling algorithm, a plurality of synthetic model input datasets based on providing a model input dataset to the resampling algorithm;
based on generating the plurality of synthetic model input datasets:
(i) providing, as model input, each model input data sample of the plurality of synthetic model input datasets to the incumbent automated decisioning model and the candidate disparity-mitigating alternative model;
(ii) computing, for each synthetic model input dataset, a model performance efficacy metric and a model fairness efficacy metric for the incumbent automated decisioning model based on individually assessing model output data of the incumbent automated decisioning model that corresponds to each respective synthetic model input dataset of the plurality of synthetic model input datasets;
(iii) computing, for each synthetic model input dataset, a model performance efficacy metric and a model fairness efficacy metric for the candidate disparity-mitigating alternative model based on individually assessing model output data of the candidate disparity-mitigating alternative model that corresponds to each respective synthetic model input dataset of the plurality of synthetic model input datasets; and
(iv) computing, for the candidate disparity-mitigating alternative model, a disparity-mitigating model viability score based on performing (1) a first set of pairwise assessments between the model performance efficacy metric computed for the incumbent automated decisioning model and the model performance efficacy metric computed for the candidate disparity-mitigating alternative model across the plurality of synthetic model input datasets and (2) a second set of pairwise assessments between the model fairness efficacy metric computed for the incumbent automated decisioning model and the model fairness efficacy metric computed for the candidate disparity-mitigating alternative model across the plurality of synthetic model input datasets;
automatically generating, via the one or more computers, one or more pieces of evidence that include rationale describing a reason that the candidate disparity-mitigating alternative model outperforms the incumbent automated decisioning model; and
displaying, via a graphical user interface, the one or more pieces of evidence and a representation of the candidate disparity-mitigating alternative model in association with the disparity-mitigating model viability score based on computing the disparity-mitigating model viability score for the candidate disparity-mitigating alternative model, wherein the computer-implemented method further includes:
receiving, via the graphical user interface, a user input selecting the candidate disparity-mitigating alternative model based at least on the displaying via the graphical user interface relating to the one or more pieces of evidence; and
in response to receiving the user input, replacing the incumbent automated decisioning model with the candidate disparity-mitigating alternative model within the automated decisioning system.