US 11,960,981 B2
Systems and methods for providing machine learning model evaluation by using decomposition
Douglas C. Merrill, Los Angeles, CA (US); Michael Edward Ruberry, Los Angeles, CA (US); Ozan Sayin, Los Angeles, CA (US); Bojan Tunguz, Greencastle, IN (US); Lin Song, Sugarland, TX (US); Esfandiar Alizadeh, Venice, CA (US); Melanie Eunique DeBruin, Northridege, CA (US); Yachen Yan, Los Angeles, CA (US); Derek Wilcox, Los Angeles, CA (US); John Candido, Burbank, CA (US); Benjamin Anthony Solecki, Los Angeles, CA (US); Jiahuan He, Los Angeles, CA (US); Jerome Louis Budzik, Altadena, CA (US); Armen Avedis Donigian, Los Angeles, CA (US); Eran Dvir, Valley Village, CA (US); Sean Javad Kamkar, Los Angeles, CA (US); Vishwaesh Rajiv, Los Angeles, CA (US); and Evan George Kriminger, Los Angeles, CA (US)
Assigned to ZESTFINANCE, INC., Burbank, CA (US)
Filed by ZestFinance, Inc., Los Angeles, CA (US)
Filed on Mar. 8, 2019, as Appl. No. 16/297,099.
Claims priority of provisional application 62/641,176, filed on Mar. 9, 2018.
Prior Publication US 2019/0279111 A1, Sep. 12, 2019
Int. Cl. G06N 20/20 (2019.01); B60Q 9/00 (2006.01); G06N 5/01 (2023.01)
CPC G06N 20/20 (2019.01) [B60Q 9/00 (2013.01); G06N 5/01 (2023.01)] 21 Claims
OG exemplary drawing
 
1. A method comprising:
receiving an observation input data set of an ensemble model and a corresponding ensemble model output, continuous model output, and tree model output, the observation input data set being associated with a rejection;
generating a first decomposition for the continuous model output relative to a baseline population by using a gradient interface of the continuous model and a second decomposition for the tree model output relative to the baseline population by using tree structure information for the tree model;
generating a third decomposition for the ensemble model output relative to the baseline population by combining the first and second decompositions in accordance with an ensemble process identified by ensemble information for the ensemble model, wherein the first and second decompositions are generated using one or more baseline input data sets representative of the baseline population;
selecting rejection explanation information for the observation input data set based on the third decomposition; and
providing the selected rejection explanation information to an operator device.