	US 12,236,353 B2
	Latent-space misalignment measure of responsible AI for machine learning models
Scott M. Zoldi, San Diego, CA (US); Jeremy Schmitt, Encinitas, CA (US); and Qing Liu, San Diego, CA (US)
Assigned to FAIR ISAAC CORPORATION, Minneapolis, MN (US)
Filed by FAIR ISAAC CORPORATION, Roseville, MN (US)
Filed on Dec. 14, 2020, as Appl. No. 17/121,594.
Prior Publication US 2022/0188644 A1, Jun. 16, 2022
Int. Cl. G06N 3/088 (2023.01); G06N 3/045 (2023.01)

CPC G06N 3/088 (2013.01) [G06N 3/045 (2023.01)]

13 Claims

1. A computer-implemented method for providing improved insights about misalignment in latent space of a machine learning model, a first artificial neural network comprising a first weight matrix and an aggregate artificial neural network associated with an aggregate data set, a second artificial neural network comprising a second weight matrix and a slice artificial neural network associated with a subset of the aggregate dataset, the method comprising:

initializing, by one or more programmable processors, the second weight matrix of the second artificial neural network based on the first weight matrix of the first artificial neural network using a first training set, the second weight matrix optimized via back-propagation based on training the second artificial neural network based on the subset of the aggregate dataset comprising a data slice of the aggregated data set;

applying, by the one or more programmable processors, transfer learning between the first artificial neural network and the second artificial neural network, the first artificial neural network including first hidden nodes defining a first latent space from the first weight matrix, the second artificial neural network including transfer learned second hidden nodes defining a second latent space based on the second weight matrix;

comparing, by the one or more programmable processors, the first latent space with the second latent space to determine a statistical distance measurement between the first latent space and the second latent space, the comparing comprising applying a normalization formula to determine a distance measure for latent space misalignment, the distance measure indicating a measure of distance between the first latent space and the second latent space determined by a statistical measure distance between at least one hidden node of the second artificial neural network and at least one hidden node of the aggregate artificial neural network based on a distribution of hidden node activation values on data slice hidden node activation values of the aggregate artificial neural network;

determining, by the one or more programmable processors and responsive to the comparing, a first score indicating alignment of the first latent space and the second latent space; and

determining, by the one or more programmable processors and responsive to the first score satisfying a threshold, whether there is a latent space misalignment between the first latent space and the second latent space by multiplying the distance measure by an absolute value of a corresponding hidden-to-output weight value in the aggregate artificial neural network.