| CPC G06N 20/20 (2019.01) | 18 Claims |
|
1. A method of mitigating output bias in a computer-based decisioning system caused by an operation of an incumbent decisioning model, the method comprising:
at a model disparity mitigation service implemented by a network of distributed computers:
sourcing, from a memory, the incumbent decisioning model of a computer-based decisioning system used in a production environment of a subscriber;
identifying a plurality of distinct candidate disparity-mitigating decisioning models for mitigating an output bias of the computer-based decisioning system;
executing, by one or more computer processors, output bias testing of the incumbent decisioning model and the plurality of distinct candidate disparity-mitigating decisioning models using a corpus of computer-generated synthetic data samples as bias testing input, wherein executing the output bias testing includes:
computing, for each of the incumbent decisioning model and the plurality of distinct candidate disparity-mitigating decisioning models, at least a first model performance metric associated with a first objective function and at least a second model performance metric associated with a second objective function based on an input of the corpus of computer-generated synthetic data samples to the incumbent decisioning model and the plurality of distinct candidate disparity-mitigating decisioning models;
computing, by the one or more computers, a two-dimensional value for each predictive output of the incumbent decisioning model and the plurality of distinct candidate disparity-mitigating decisioning models based on the first model performance metric and the second model performance metric;
executing, by the one or more computers, a pairwise assessment between a first plurality of two-dimensional values for the incumbent decisioning model and a second plurality of two-dimensional values for at least one of the plurality of distinct candidate disparity-mitigating decisioning models, wherein the pairwise assessment includes identifying as a target pair a first two dimensional value of the first plurality of two-dimensional values and a second two-dimensional value of the second plurality of two-dimensional values that share a same bias testing input comprising a given synthetic data sample of the corpus of computer-generated synthetic data samples that was commonly inputted into the incumbent decisioning model and the at least one of the plurality of distinct candidate disparity-mitigating decisioning models;
generating, via a graphical user interface, a graphical visualization, of the first plurality of two-dimensional values and the second plurality of two-dimensional values, wherein the graphical visualization overlays bias and performance trade-offs for the incumbent decisioning model and the at least one of the plurality of distinct candidate disparity-mitigating decisioning models;
identifying a given candidate disparity-mitigating decisioning model of the plurality of distinct candidate disparity-mitigating decisioning models that, when executed, mitigates the output bias in the computer-based decisioning system; and
outputting, by the one or more computer processors, a recommendation for adapting the computer-based decisioning system by replacing the incumbent decisioning model with the given candidate disparity-mitigating decisioning model;
receiving, via the graphical user interface, a user input selecting the given candidate disparity-mitigating decisioning model from the plurality of distinct candidate disparity-mitigating decisioning models; and
in response to receiving the user input selection, replacing the incumbent decisioning model with the given candidate disparity-mitigating decisioning model thereby improving decisioning outputs of the computer-based decisioning system.
|