US 11,995,518 B2
Machine learning model understanding as-a-service
Eric Zavesky, Austin, TX (US); David Crawford Gibbon, Lincroft, NJ (US); Lee Begeja, Gillette, NJ (US); Paul Triantafyllou, Watchung, NJ (US); and Behzad Shahraray, Holmdel, NJ (US)
Assigned to AT&T Intellectal P Property I, L.P., Atlanta, GA (US)
Filed by AT&T Intellectual Property I, L.P., Atlanta, GA (US)
Filed on Dec. 20, 2017, as Appl. No. 15/849,115.
Prior Publication US 2019/0188605 A1, Jun. 20, 2019
Int. Cl. G06N 20/00 (2019.01)
CPC G06N 20/00 (2019.01) 20 Claims
OG exemplary drawing
 
1. A model understanding as-a-service system comprising:
a processor; and
a memory having instructions stored thereon that, when executed by the processor, cause the processor to perform operations, the operations comprising
receiving, from a user system, a service request comprising a machine learning model, wherein the service request instructs the model understanding as-a-service system to conduct an analysis of the machine learning model to identify a feature bias exhibited by the machine learning model, wherein the service request further instructs the model understanding as-a-service system to apply a compliance policy for the analysis, and wherein the compliance policy comprises a user-imposed compliance policy,
conducting the analysis of the machine learning model in accordance with the service request to identify the feature bias exhibited by the machine learning model, wherein conducting the analysis comprises
determining a plurality of perturbations of a plurality of features of the machine learning model,
applying the plurality of perturbations of the plurality of features of the machine learning model for a plurality of samples,
executing, on the model understanding as-a-service system, the machine learning model over the plurality of samples as inputs to obtain a plurality of outputs, wherein an effect that each perturbation of the plurality of perturbations has on an output of the machine learning model is based upon a correlation of the plurality of outputs, and
identifying the feature bias based upon the effect that each perturbation of the plurality of perturbations has on the output of the machine learning model,
training a second machine learning model with a first training data set, wherein the first training data set is created by removing from a second training data set at least one feature of the plurality of features that is associated with the feature bias, wherein the second training data set was used to train the machine learning model;
computing a performance loss of the machine learning model based upon a difference between a first accuracy of the machine learning model as compared to a second accuracy of the second machine learning model;
compiling results of the analysis of the machine learning model in accordance with the service request, wherein the results of the analysis identify the feature bias and the performance loss that is computed,
creating a service response comprising the results of the analysis, and
providing the service response to the user system.