	US 11,797,888 B2
	System and method for automatically retraining machine learning models to predict bias in a data input
Austin Walters, Savoy, IL (US); Mark Watson, Urbana, IL (US); Jeremy Goodsitt, Champaign, IL (US); and Anh Truong, Champaign, IL (US)
Assigned to Capital One Services, LLC, McLean, VA (US)
Filed by Capital One Services, LLC, McLean, VA (US)
Filed on Jul. 2, 2021, as Appl. No. 17/366,554.
Application 17/366,554 is a continuation of application No. 16/731,575, filed on Dec. 31, 2019, granted, now 11,062,234, issued on Jul. 13, 2021.
Prior Publication US 2021/0334707 A1, Oct. 28, 2021
Int. Cl. G06N 20/00 (2019.01); G06F 16/28 (2019.01); G06N 7/01 (2023.01)

CPC G06N 20/00 (2019.01) [G06F 16/285 (2019.01); G06N 7/01 (2023.01)]

20 Claims

1. A method, comprising:

receiving concurrently into a parallel configuration of a classification machine learning model and a bias filter machine learning model a data input for classification into at least one data category of a plurality of data categories;

wherein the data input comprises initial data and the plurality of data categories;

generating concurrently a classification output by the classification machine learning model and a bias confidence probability output by the bias filter machine learning model;

inputting the classification output and the bias confidence probability output into a gate machine learning model;

wherein the classification output comprises:

a classification of the initial data in the at least one data category from the plurality of data categories to form a classified data, and

a classification confidence probability in the classification of the initial data in the classified data;

wherein the bias confidence probability output comprises a bias confidence probability that the classification of the initial data in the classified data is related to at least one bias characteristic;

training incrementally the classification, bias filter, and gate machine learning models in an iterative manner with feedback using the data input, the classification output, and the bias confidence probability output;

generating a classification outcome of the classified data by the gate machine learning model based on the classification output and the bias confidence probability output;

wherein the classification outcome is one of a category of:

i) biased

ii) potentially biased, or

iii) unbiased;

retraining, for each of the at least one bias characteristic wherein the classification outcome is biased or potentially biased, for at least one subsequent iteration of the incremental iterative training with feedback of the classification, bias filter, and gate machine learning models to update the classification, bias filter, and gate machine learning models until the bias confidence probability is below a predefined bias threshold by blocking the classified data that is biased and adding into the data input at least:

i) the classification output and

ii) the bias confidence probability output; and

outputting the classification outcome of the classified data to a computing device associated with a user when the classification outcome of the classified data is unbiased or potentially biased and blocking an output of the classified data when it is determined by the classification, bias filter, and gate machine learning models that the classification outcome of the classified data is biased.