US 11,893,459 B1
Artificial intelligence labeling platform for secure data including observation validation
Christopher Jacoby, Oakland, CA (US); Thomas Chase Corcoran, San Jose, CA (US); and Adrian Lam, South Francisco, CA (US)
Assigned to CHANGE HEALTHCARE HOLDINGS LLC, Nashville, TN (US)
Filed by Change Healthcare Holdings LLC, Nashville, TN (US)
Filed on Sep. 24, 2020, as Appl. No. 17/030,881.
Int. Cl. G06N 20/00 (2019.01); G06F 16/28 (2019.01); G06F 21/62 (2013.01); G06F 21/30 (2013.01); G06F 16/23 (2019.01); G06N 5/04 (2023.01); G06F 16/22 (2019.01)
CPC G06N 20/00 (2019.01) [G06F 16/2228 (2019.01); G06F 16/2379 (2019.01); G06F 16/285 (2019.01); G06F 21/30 (2013.01); G06F 21/6245 (2013.01); G06N 5/04 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method, comprising:
identifying using one or more processors, records in a database for labeling;
verifying, using the one or more processors, that a labeling entity is authorized to view a first one of the records in the database;
presenting, using the one or more processors, the first one of the records in the database to the labeling entity;
receiving using the one or more processors, a first observation on an information source in the first one of the records from the first labeling entity, the first observation having one of a plurality of observation types associated therewith, the plurality of observation types comprising a validation observation type in which the first observation comprises a confirmation of whether a second observation on the information source in the first one of the records from another labeling entity is accurate and an edit for the second observation when the second observation is confirmed as inaccurate;
aggregating the first observation with the second observation to generate an aggregated observation on the information source;
updating, using the one or more processors, the first one of the records in the database with the aggregated observation on the information source;
initiating using the one or more processors and based on the updated first one of the records, training of an Artificial Intelligence (AI) system to generate an AI model;
evaluating, using the one or more processors, an accuracy of the AI model;
prioritizing, using the one or more processors, the records in the database for labeling based on the accuracy of the AI model; and
presenting, using the one or more processors, a second one of the records in the database to the labeling entity based on a priority of the second one of the records in the database.