US 11,923,094 B2
Monitoring predictive models
Kanakasabha Kailasam, Olathe, KS (US); and Sasanka Are, Topeka, KS (US)
Assigned to Cerner Innovation, Inc., Kansas City, KS (US)
Filed by CERNER INNOVATION, INC., Kansas City, KS (US)
Filed on Dec. 23, 2021, as Appl. No. 17/560,653.
Application 17/560,653 is a continuation of application No. 15/386,906, filed on Dec. 21, 2016, granted, now 11,244,764.
Prior Publication US 2022/0115144 A1, Apr. 14, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G16H 50/50 (2018.01); G16H 50/20 (2018.01); G16H 50/70 (2018.01)
CPC G16H 50/50 (2018.01) [G16H 50/20 (2018.01); G16H 50/70 (2018.01)] 20 Claims
OG exemplary drawing
 
1. One or more computer-readable storage devices having computer-executable instructions embodied thereon that, when executed, facilitate a method for assessing performance of a predictive model on a population of individuals, the method comprising:
extracting medical information from a set of electronic medical records associated with the population of individuals having a common medical condition;
generating, by the predictive model, a plurality of numerical condition scores and a plurality of risk levels corresponding to the plurality of numerical condition scores based on the set of electronic medical records;
determining, based on the plurality of numerical condition scores and the plurality of risk levels, a plurality of monitoring values computed using the predictive model, the plurality of monitoring values including at least a first monitoring value and a second monitoring value for each individual within the population;
determining, from the medical information, a plurality of actions taken, each action taken within the plurality of actions taken being in response to the first monitoring value and being performed before computation of the second monitoring value for each individual, each action taken corresponding to a recommended action generated by the predictive model in response to the first monitoring value, wherein the recommended action is generated prior to a first discharge of the individual from a healthcare facility;
determining a plurality of time periods, wherein each time period occurs prior to the first discharge;
associating each action taken for each individual with a trend in monitoring values and an amount of time between when the recommended action is provided and when the action is taken, the trend being based on at least the first monitoring value and the second monitoring value for each individual;
generating an assessment of the predictive model based on the trend associated with the action taken for each individual and the amount of time;
adjusting the predictive model based on the generated assessment;
determining, using the adjusted predictive model, a first time taken by the population of individuals to reach from a first risk level of the plurality of risk levels to a second risk level of the plurality of risk levels, in response to the plurality of actions taken for the medical condition;
determining, using the adjusted predictive model and in response to the action for the medical condition, a second time taken by an individual to reach, from the first risk level of the plurality of risk levels, to a second risk level of the plurality of risk levels; and
generating an alert, using the adjusted predictive model, when the second time taken by the individual to reach from the first risk level of the plurality of risk levels to a second risk level of the plurality of risk levels deviates from the first time taken by the population.