US 11,729,194 B2
Automatic model monitoring for data streams
Marco Oliveira Pena Sampaio, Vila Nova de Gaia (PT); Fábio Hernäni dos Santos Costa Pinto, Oporto (PT); Pedro Gustavo Santos Rodrigues Bizarro, Lisbon (PT); Pedro Cardoso Lessa e Silva, Oporto (PT); Ana Margarida Caetano Ruela, Lisbon (PT); Miguel Ramos de Araújo, Oporto (PT); and Nuno Miguel Lourenço Diegues, Lisbon (PT)
Assigned to Feedzai—Consultadoria e Inovação Tecnológica, S.A.
Filed by Feedzai—Consultadoria e Inovaçâo Tecnológica, S.A., Coimbra (PT)
Filed on Jun. 10, 2022, as Appl. No. 17/838,079.
Application 17/838,079 is a continuation of application No. 16/667,674, filed on Oct. 29, 2019, granted, now 11,451,568.
Claims priority of provisional application 62/847,101, filed on May 13, 2019.
Prior Publication US 2022/0382861 A1, Dec. 1, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. H04N 21/433 (2011.01); H04L 9/40 (2022.01); H04L 43/16 (2022.01); H04L 41/0604 (2022.01); G06F 16/22 (2019.01); G06N 20/20 (2019.01); G06N 5/04 (2023.01); G06Q 20/40 (2012.01); G06N 20/00 (2019.01); G06F 16/2458 (2019.01); G06F 21/55 (2013.01); G06N 5/01 (2023.01); G06F 3/0484 (2022.01); G06F 21/62 (2013.01)
CPC H04L 63/1425 (2013.01) [G06F 16/22 (2019.01); G06F 16/2474 (2019.01); G06F 21/554 (2013.01); G06N 5/01 (2023.01); G06N 5/04 (2013.01); G06N 20/00 (2019.01); G06N 20/20 (2019.01); G06Q 20/4016 (2013.01); H04L 41/0604 (2013.01); H04L 43/16 (2013.01); H04L 63/0428 (2013.01); G06F 3/0484 (2013.01); G06F 21/6245 (2013.01); G06F 2221/034 (2013.01); H04L 63/1416 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A method comprising:
receiving an input dataset;
using a machine learning model to determine a model score for each data record of at least a portion of the input dataset;
determining monitoring values including by summing individual divergence contributions for each bin of a group of bins, wherein each monitoring value of at least a portion of the determined monitoring values is associated with a measure of similarity comparing (i) a histogram of model scores for those data records of the input dataset within a corresponding moving reference window and (ii) a histogram of model scores for those data records of the input dataset within a corresponding moving target window; and
outputting the determined monitoring values.