US 12,339,928 B2
Decreasing error in a machine learning model based on identifying reference and monitored groups of the machine learning model
Ravi Chandra Chamarthy, Hyderabad (IN); Manish Anand Bhide, Hyderabad (IN); Madhavi Katari, Hyderabad (IN); and Arunkumar Kalpathi Suryanarayanan, Chennai (IN)
Assigned to International Business Machines Corporation, Armonk, NY (US)
Filed by International Business Machines Corporation, Armonk, NY (US)
Filed on Nov. 20, 2020, as Appl. No. 16/953,587.
Prior Publication US 2022/0164606 A1, May 26, 2022
Int. Cl. G06N 20/00 (2019.01); G06F 11/30 (2006.01); G06F 18/21 (2023.01)
CPC G06F 18/217 (2023.01) [G06F 11/302 (2013.01); G06N 20/00 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A method, in a data processing system comprising a processor and memory, the memory comprising instructions executed by the processor to cause the processor to implement a machine learning model data quality improvement detection tool that identifies an accurate reference group and an accurate monitored group of a machine learning model, the method comprising:
monitoring, by the machine learning model data quality improvement detection tool, a behavior of the machine learning model for a predetermined time frame;
comparing, by the machine learning model data quality improvement detection tool, a determined fairness metric that utilizes a percentage of favorable outcomes of a user-defined monitored group and a percentage of favorable outcomes of a user-defined reference group to a pre-defined fairness threshold;
responsive to the determined fairness metric failing to meet the pre-defined fairness threshold, modifying, by the machine learning model data quality improvement detection tool, the user-defined monitored group to include a first portion of the user-defined reference group thereby forming a modified monitored group and a modified reference group;
comparing, by the machine learning model data quality improvement detection tool, a newly determined fairness metric that utilizes a percentage of favorable outcomes of the modified monitored group and a percentage of favorable outcomes of the modified reference group to the pre-defined fairness threshold; and
responsive to the newly determined fairness metric meeting the pre-defined fairness threshold, identifying, by the machine learning model data quality improvement detection tool, the modified monitored group including the first portion of the user-defined reference group as a new monitored group and the modified reference group without the first portion of the user-defined reference group as a new reference group.