| CPC G06N 3/08 (2013.01) [G06N 3/04 (2013.01)] | 20 Claims |
|
1. A method, comprising:
determining, in a machine learning system implemented utilizing processor and memory resources of at least one computing device, a performance metric of a machine learning model at runtime, wherein the machine learning model is a trained machine learning model, trained using a training data set;
determining, in the machine learning system, a homogeneity degree between a verification data set processed by the machine learning model at runtime and the training data set used to train the machine learning model, the verification data set comprising a plurality of verification data samples, each of the verification data samples including a corresponding model input processed by the machine learning model at runtime, wherein the homogeneity degree is determined by performing clustering between multiple feature representations of the training data set to obtain a plurality of clusters, determining feature representations of the verification data samples, computing differences between the feature representations of the verification data samples and one or more characteristics of the clusters, and determining the homogeneity degree based on the computed differences;
determining, in the machine learning system, a type of a conceptual drift of the machine learning model based on the performance metric and the homogeneity degree;
performing, in the machine learning system, an update of the machine learning model based on the type of the conceptual drift, wherein the update comprises a partial update or a global update;
executing the updated machine learning model in the machine learning system to process one or more additional data sets; and
repeating the determining of the performance metric of the machine learning model at runtime, the determining of the homogeneity degree, the determining of the type of conceptual drift, the performing of the update and the executing of the updated machine learning model in each of a plurality of iterations to provide automated adaptive updating of the machine learning model in the machine learning system, with different ones of the partial update and the global update being performed in different ones of the iterations based on the type of conceptual drift determined in each such iteration;
wherein determining a type of conceptual drift of the machine learning model based on the performance metric and the homogeneity degree comprises distinguishing between multiple distinct types of conceptual drift including at least conceptual drift in an output of the machine learning model, conceptual drift in an input of the machine learning model, and conceptual drift in a relationship between the input and the output of the machine learning model; and
wherein performing an update of the machine learning model based on the type of the conceptual drift comprises updating different portions of the machine learning model utilizing respective distinct update processes responsive to determining respective different ones of the multiple distinct types of conceptual drift, a first one of the update processes applied responsive to determining a first type of conceptual drift comprising locally updating an output layer of a neural network of the machine learning model through calibration performed in a model calibrator deployed after the output layer of the neural network and configured to automatically adjust outputs of the output layer responsive to detection of the first type of conceptual drift, a second one of the update processes applied responsive to determining a second type of conceptual drift comprising locally updating the output layer of the neural network through parameter adjustment in which one or more parameters of the output layer but no hidden layers of the neural network are automatically updated responsive to detection of the second type of conceptual drift, and a third one of the update processes applied responsive to determining a third type of conceptual drift comprising globally retraining the neural network including the output layer and at least one or more hidden layers of the neural network, the global retraining automatically updating one or more parameters of the output layer and the one or more hidden layers of the neural network responsive to detection of the third type of conceptual drift.
|