US 11,922,435 B2
Detecting and reducing bias (including discrimination) in an automated decision making process
Gabriel M. Silberman, Austin, TX (US); Michael Louis Roberts, Austin, TX (US); Jean Belanger, Austin, TX (US); and Karen Bennet, Toronto (CA)
Assigned to CEREBRI AI INC., Austin, TX (US)
Filed by Cerebri AI Inc., Austin, TX (US)
Filed on Sep. 1, 2020, as Appl. No. 17/009,482.
Application 17/009,482 is a continuation of application No. 15/595,220, filed on May 15, 2017, granted, now 10,861,028.
Claims priority of provisional application 62/337,311, filed on May 16, 2016.
Prior Publication US 2021/0056569 A1, Feb. 25, 2021
Int. Cl. G06N 20/00 (2019.01); G06F 18/2321 (2023.01); G06F 18/2415 (2023.01); G06N 5/025 (2023.01); G06N 7/01 (2023.01); G06Q 10/067 (2023.01); G06Q 30/0201 (2023.01); G06F 8/35 (2018.01)
CPC G06Q 30/0201 (2013.01) [G06F 18/2321 (2023.01); G06F 18/24155 (2023.01); G06N 5/025 (2013.01); G06N 7/01 (2023.01); G06N 20/00 (2019.01); G06Q 10/067 (2013.01); G06F 8/35 (2013.01)] 25 Claims
OG exemplary drawing
 
1. One or more tangible, non-transitory, machine-readable media storing instructions that, when executed by a computer system, effectuate operations comprising:
obtaining, with a computer system, access to a first machine-learning model,
the first machine-learning model being responsive to inputs corresponding to a first set of attributes,
the first set of attributes being attributes of entities in a population of entities, and
the first machine-learning model being trained on a first set of data to make decisions about the entities or otherwise produce outputs upon which such decisions are made;
obtaining, with the computer system, a second set of attributes of entities in the population of entities, the second set of attributes being designated as attributes for which correlation with output of the first machine-learning model is to be reduced or eliminated in the decisions;
obtaining, with the computer system, access to a second machine-learning model trained on a second set of data to make decisions about the entities or otherwise produce outputs upon which such decisions are made, outputs of the second machine-learning model being less correlated with the second set of attributes than outputs of the first machine-learning model;
detecting, with the computer system, a correlation of outputs of the first machine-learning model with a member of the second set of attributes by:
obtaining a first set of inputs corresponding to a first entity,
obtaining a first output of the first machine-learning model based on the first set of inputs corresponding to the first entity,
obtaining a second output of the second machine-learning model based on the first set of inputs corresponding to the first entity, and
determining that the first output differs from the second output by more than a first threshold amount; and
storing, with the computer system, a record of detecting the correlation of outputs of the first machine-learning model with the member of the second set of attributes in memory, wherein the operations further comprise:
adjusting a value indicative of a count of detections of bias of the first machine-learning model in response to detecting the correlation of outputs of the first machine-learning model with the member of the second set of attributes;
determining that the value satisfies the second threshold, and
in response to determining that the value satisfies the second threshold, modifying or creating a modified version of the first machine-learning model or the second machine-learning model.