US 11,841,941 B2
Systems and methods for accelerated detection and replacement of anomalous machine learning-based ensembles and intelligent generation of anomalous artifacts for anomalous ensembles
Pradhan Bagur Umesh, San Francisco, CA (US); Yuan Zhuang, San Francisco, CA (US); Hui Wang, San Francisco, CA (US); Nicholas Benavides, San Francisco, CA (US); Chang Liu, San Francisco, CA (US); Yanqing Bao, San Francisco, CA (US); and Wei Liu, San Francisco, CA (US)
Assigned to Sift Science, Inc., San Francisco, CA (US)
Filed by Sift Science, Inc., San Francisco, CA (US)
Filed on Jun. 16, 2023, as Appl. No. 18/210,985.
Application 18/210,985 is a continuation of application No. 17/963,365, filed on Oct. 11, 2022, granted, now 11,720,668.
Claims priority of provisional application 63/254,464, filed on Oct. 11, 2021.
Prior Publication US 2023/0325494 A1, Oct. 12, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 21/55 (2013.01); G06N 20/20 (2019.01); G06N 5/04 (2023.01)
CPC G06F 21/55 (2013.01) [G06N 5/04 (2013.01); G06N 20/20 (2019.01); G06F 2221/034 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A method for accelerated anomaly detection and mitigating an anomalous behavior of an ensemble of machine learning models, the method comprising:
detecting, by one or more computers, an anomalous behavior of an ensemble of machine learning models, wherein the anomalous behavior relates to an unexpected deviation in one or more inferences of the ensemble of machine learning models;
in response to the detecting, identifying an errant machine learning model of the ensemble of machine learning models contributing to the anomalous behavior;
configuring a candidate machine learning model as a likely replacement of the errant machine learning model based on the identification of the errant machine learning model;
calculating one or more efficacy metrics for a simulated combination of the machine learning model with the ensemble of machine learning models; and
mitigating the anomalous behavior by updating the ensemble of machine learning models with the candidate machine learning model based on the one or more efficacy metrics of the simulated combination satisfying one or more efficacy benchmarks.