US 11,676,077 B2
Model interpretation
Mark Chan, San Jose, CA (US); Navdeep Gill, San Jose, CA (US); and Patrick Hall, Washington, DC (US)
Assigned to H2O.ai Inc., Mountain View, CA (US)
Filed by H2O.ai Inc., Mountain View, CA (US)
Filed on Apr. 20, 2018, as Appl. No. 15/959,040.
Prior Publication US 2019/0325335 A1, Oct. 24, 2019
Int. Cl. G06N 20/20 (2019.01); G06N 7/01 (2023.01); G06N 20/00 (2019.01); G06F 18/23213 (2023.01); G06F 18/243 (2023.01)
CPC G06N 20/20 (2019.01) [G06F 18/23213 (2023.01); G06F 18/24323 (2023.01); G06N 7/01 (2023.01); G06N 20/00 (2019.01)] 19 Claims
OG exemplary drawing
 
1. A method, comprising:
receiving an indication of a selection of an entry associated with a linear surrogate machine learning model, wherein the linear surrogate machine learning model approximates an output associated with a machine learning model;
in response to the received indication of the selection of the entry associated with the linear surrogate machine learning model, dynamically updating one or more interpretation views associated with one or more other surrogate machine learning models based on the selected entry, wherein dynamically updating one or more interpretation views associated with one or more other surrogate machine learning models includes updating a first non-linear surrogate machine learning model comprising a decision tree surrogate model that indicates a path associated with the selected entry associated with the linear surrogate machine learning model through the updated decision tree surrogate model, wherein a width associated with the selected entry is updated to indicate a frequency with which the path associated with the selected entry is used by the updated decision tree surrogate model;
determining whether an output associated with the linear surrogate machine learning model correlates with an output associated with the first non-linear surrogate model; and
retraining the linear surrogate machine learning model and/or the first non-linear surrogate machine learning model in response to a determination that the output associated with the linear surrogate machine learning model does not correlate with the output associated with the first non-linear surrogate model for more than a threshold number of entries.