| CPC G06N 20/20 (2019.01) [G06N 5/04 (2013.01)] | 13 Claims |
|
1. A computer-implemented method for continuously evaluating a learning model used for monitoring an industrial process involving an IT computing infrastructure and predicting the occurrence of an anomaly, said learning model being one of a plurality of learning models, said learning models being defined by parameters and hyperparameters, said parameters of said learning models having been learnt during a training process based on training datasets comprising data measurements related to industrial machines involved in said industrial process, said data measurements comprising performance indicators of said IT computing infrastructure, received from industrial sensors, comprising computing probes and physical sensors selected from the group consisting of a temperature sensor, a humidity sensor and a power consumer sensor, for training the learning models to predict anomalies in the industrial process, said performance indicators being selected from the group consisting of resource usage, event history, software errors, hardware errors, response times, application traffic, service load, network traffic, file modifications, number of users of a service, number of sessions, number of processes, temperature values, humidity values, power consumption, CPU usage, memory usage, server response time, number of slow pages and number of transactions, said hyperparameters having values that are set before the learning process begins, said method being implemented by a model selection module of a computing device, said computing device further comprising a processor operably coupled to a non-transitory computer readable storage medium storing a model selection module and a model repository including a plurality of series of instructions each corresponding to one learning model of said plurality of learning models and each including said parameters and hyperparameters, said processor being configured to execute said model selection module and to run said plurality of learning models, said method comprising:
receiving an input dataset used by the learning model being evaluated, said input dataset comprising data measurements related to said industrial machines, from said industrial sensors, and storing said input dataset in said memory;
evaluating prediction performance of each learning model of said plurality of learning models on the input dataset, by determining model prediction performance indicators;
an evaluation processing including a step of evaluating a classification, for providing, for each learning model of said plurality of learning models, at least one classification performance value;
pre-selecting a learning model of said plurality of learning models, a prediction performance value and the classification value of which are greater than first predetermined threshold values;
varying hyperparameters of the selected learning model, based on hyperparameter optimization indicators, to produce optimized hyperparameters that optimize a cost function representative of a most accurate prediction of behavior of the industrial process;
selecting the pre-selected learning model associated with the optimized hyperparameters as suitable to process the input data set, when a prediction performance value and the classification value are greater than predetermined second threshold values and, for each optimized hyperparameter, the cost function of the selected model is greater than a predetermined performance threshold value; and
submitting the selected learning model associated with the optimized hyperparameters to a user in order to adapt or replace the learning model being evaluated.
|