US 11,748,638 B2
Machine learning model monitoring
Rafal Bigaj, Cracow (PL); Lukasz G. Cmielowski, Cracow (PL); Wojciech Sobala, Cracow (PL); and Maksymilian Erazmus, Zasów (PL)
Assigned to International Business Machines Corporation, Armonk, NY (US)
Filed by International Business Machines Corporation, Armonk, NY (US)
Filed on Jul. 22, 2020, as Appl. No. 16/935,670.
Prior Publication US 2022/0027749 A1, Jan. 27, 2022
Int. Cl. G06N 5/04 (2023.01); G06N 20/00 (2019.01)
CPC G06N 5/04 (2013.01) [G06N 20/00 (2019.01)] 16 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
receiving a dataset for processing by a machine learning model (MLM);
receiving a scoring payload comprising a set of information related to operation of the MLM using the dataset;
determining a set of features of the machine learning model by analyzing the scoring payload;
determining prediction results that are predicted to be produced by the MLM for the scoring payload by analyzing fields within the scoring payload; and
structuring the scoring payload in accordance with the set of features such that the structured scoring payload is ready for analysis for a monitor of the machine learning model, wherein the structuring utilizes a technique selected from the group consisting of a classification technique, a correlation technique, and a direct value identification technique;
wherein:
the set of features is determined by a controller that includes a second MLM that utilizes a multioutput multiclass classification technique;
the set of features comprises:
a learning framework of the machine learning model;
a service provider of the machine learning model;
a problem type for the scoring payload; and
whether a result of the scoring payload is an encoded target or a decoded target;
the method further comprising:
sending the structured scoring payload from the controller to the monitor for determining a health of the MLM.