US 11,989,112 B2
Model validation based on sub-model performance
Uttam Ramamurthy, Bangalore (IN); Ashwin Chhetri, Darjeeling (IN); Pankaj Saxena, Pune (IN); and Rahul Jain V, Bangalore (IN)
Assigned to Cerner Innovation, Inc., Kansas City, MO (US)
Filed by CERNER INNOVATION, INC., North Kansas City, MO (US)
Filed on Jul. 11, 2022, as Appl. No. 17/811,787.
Claims priority of provisional application 63/294,533, filed on Dec. 29, 2021.
Prior Publication US 2023/0205668 A1, Jun. 29, 2023
Int. Cl. G06F 11/34 (2006.01); G06F 9/445 (2018.01); G06N 20/00 (2019.01)
CPC G06F 11/3495 (2013.01) [G06F 9/44505 (2013.01); G06F 11/3428 (2013.01); G06N 20/00 (2019.01)] 34 Claims
OG exemplary drawing
 
1. One or more non-transitory media having instructions that, when executed by one or more processors, cause the one or more processors to facilitate a plurality of operations for evaluating and improving model version performance and accuracy, the operations comprising:
inputting healthcare data to a model pipeline that includes sub-models, wherein an output of one of the sub-models determines an input of another of the sub-models;
detecting, as an output from the model pipeline in response to the inputted healthcare data, a model-pipeline prediction computed based on the sub-models;
retrieving one or more files containing formatted datasets corresponding to the model pipeline, wherein at least one of the formatted datasets includes data used to generate the model-pipeline prediction;
receiving a configuration file; and
computing a plurality of sub-model performance reports (a) for a plurality of the sub-models, (b) from the formatted datasets, and (c) based on the configuration file, wherein the plurality of sub-model performance reports includes performance metrics determined for a corresponding version of the model pipeline.