US 12,282,527 B2
Determining system performance without ground truth
Dinesh C. Verma, New Castle, NY (US); and Seraphin Bernard Calo, Cortlandt Manor, NY (US)
Assigned to International Business Machines Corporation, Armonk, NY (US)
Filed by International Business Machines Corporation, Armonk, NY (US)
Filed on Sep. 1, 2020, as Appl. No. 17/008,747.
Prior Publication US 2022/0067450 A1, Mar. 3, 2022
Int. Cl. G06F 18/21 (2023.01); G06F 18/214 (2023.01); G06N 20/00 (2019.01)
CPC G06F 18/2185 (2023.01) [G06F 18/2148 (2023.01); G06N 20/00 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
requesting, by an edge node of a plurality of edge nodes, both a trained model and one or more generator models from a core node coupled to the plurality of edge nodes;
receiving, by the edge node, both the trained model and the one or more generator models from the core node coupled to the plurality of edge nodes, the trained model having been trained on training data at the core node, the one or more generator models being configured to produce synthetic training data representative of the training data and having been created at the core node, wherein the edge node receives both the trained model and the one or more generator models from the core node in response to the requesting the trained model and the one or more generator models;
executing the edge node in a real-world environment to capture testing data, wherein the executing the edge node in the real-world environment to capture the testing data comprises employing components to capture the testing data under operating conditions;
inputting, by the edge node, the testing data to the trained model to produce labeled testing data, wherein the edge node comprises a proxy model distinct from the trained model and the one or more generator models, wherein the trained model, the one or more generator models, and the proxy model are machine learning models;
training, by the edge node, the proxy model with the labeled testing data, the proxy model having a machine learning architecture corresponding to the trained model; and
inputting, by the edge node, the synthetic training data to the proxy model to produce predictions related to a performance of the trained model.