US 11,720,668 B2
Systems and methods for accelerated detection and replacement of anomalous machine learning-based digital threat scoring ensembles and intelligent generation of anomalous artifacts for anomalous ensembles
Pradhan Bagur Umesh, San Francisco, CA (US); Yuan Zhuang, San Francisco, CA (US); Hui Wang, San Francisco, CA (US); Nicholas Benavides, San Francisco, CA (US); Chang Liu, San Francisco, CA (US); Yanqing Bao, San Francisco, CA (US); and Wei Liu, San Francisco, CA (US)
Assigned to Sift Science, Inc., San Francisco, CA (US)
Filed by Sift Science, Inc., San Francisco, CA (US)
Filed on Oct. 11, 2022, as Appl. No. 17/963,365.
Claims priority of provisional application 63/254,464, filed on Oct. 11, 2021.
Prior Publication US 2023/0124621 A1, Apr. 20, 2023
Int. Cl. G06F 21/55 (2013.01); G06N 20/20 (2019.01); G06N 5/04 (2023.01)
CPC G06F 21/55 (2013.01) [G06N 5/04 (2013.01); G06N 20/20 (2019.01); G06F 2221/034 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for accelerated anomaly detection and replacement of an anomaly-experiencing machine learning-based ensemble, the method comprising:
identifying, by one or more computers, a machine learning-based digital threat scoring ensemble having an anomalous drift behavior in digital threat score inferences computed by the machine learning-based digital threat scoring ensemble for a target period;
executing, based on the identifying, a tiered anomaly evaluation for the machine learning-based digital threat scoring ensemble, wherein the tiered anomaly evaluation includes:
(a) identifying at least one machine learning-based model of the machine learning-based digital threat scoring ensemble contributing to the anomalous drift behavior, and
(b) identifying at least one feature variable of the at least one machine learning-based model contributing to the anomalous drift behavior;
generating a potential successor machine learning-based digital threat scoring ensemble to the machine learning-based digital threat scoring ensemble based on the tiered anomaly evaluation, wherein the potential successor machine learning-based digital threat scoring ensemble mitigates the anomalous drift behavior; and
replacing the machine learning-based digital threat scoring ensemble with the potential successor machine learning-based digital threat scoring ensemble based on one or more ensemble metrics computed for the potential successor machine learning-based digital threat scoring ensemble satisfying one or more efficacy benchmarks.