| CPC G06N 20/00 (2019.01) [G06N 3/08 (2013.01)] | 20 Claims |

|
1. A computer-implemented method for training models, comprising:
receiving, by a first server from a second server of a plurality of servers, inference data generated based on a first model, wherein content of data captured by one or more sensor devices is evaluated by the second server based on the first model to generate the inference data, the inference data describes identifying an object in the content of the captured data, and the second server of the plurality of servers comprises less computing resources than the first server;
determining, by the first server, a data drift in the received inference data generated based on the first model, wherein the data drift indicates a degree of inaccuracy of the inference data that identify the object in the content of the captured data;
determining that the data drift exceeds a drift threshold;
evaluating computing resources of each of the plurality of servers;
determining whether the second server has sufficient computing resources to train a second model while continuing to generate inference data;
in response to determining the second server does not have sufficient computing resources, determining at least a third server of the plurality of servers that has sufficient computing resources to train the second model;
requesting at least the third server to train the second model;
receiving the trained second model from the third server; and
causing, by the first server, the first model to be updated with the trained second model on a graphics processing unit (GPU) of the second server of the plurality of servers, thereby causing the second server to execute an inference operation on the GPU using trained second model” and replace with the first model updated with the trained second model for improved accuracy of inferencing a subsequently captured sensor data.
|