US 11,934,957 B2
Methods, systems, and apparatuses for user-understandable explainable learning models
Claudia V. Goldman-Shenhar, Mevasseret Zion (IL); and Michael Baltaxe, Raanana (IL)
Assigned to GM GLOBAL TECHNOLOGY OPERATIONS LLC, Detroit, MI (US)
Filed by GM GLOBAL TECHNOLOGY OPERATIONS LLC, Detroit, MI (US)
Filed on Sep. 17, 2020, as Appl. No. 17/023,515.
Claims priority of provisional application 63/071,135, filed on Aug. 27, 2020.
Prior Publication US 2022/0067511 A1, Mar. 3, 2022
Int. Cl. G06N 3/08 (2023.01); G06N 3/0464 (2023.01); G06N 3/082 (2023.01); G06V 30/262 (2022.01)
CPC G06N 3/082 (2013.01) [G06N 3/0464 (2023.01); G06V 30/274 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A method for constructing an explainable user output for control of a vehicle, the method comprising:
receiving input data for the vehicle of a set of features for processing at a neural network (NN) that is stored within a non-transitory computer readable storage medium of the vehicle, wherein the neural network is composed of a plurality of layers of an original classifier, wherein the original classifier has been frozen with a set of weights;
determining, via a processor of the vehicle, a semantic function to categorize a data sample with a semantic category;
determining, via the processor of the vehicle, a level of semantic accuracy for each layer of the plurality of layers of the original classifier within the neural network wherein the original classifier is a trained model;
computing, via the processor of the vehicle, a representative vector with average activations of a layer's nodes of the plurality of layers for training samples of a semantic category determined for evaluation by computing distances of samples in a test set from available layers for each semantic category;
computing, via the processor of the vehicle, a number of test samples for each layer and for each semantic category of a plurality of semantic categories, that are closest to each other in each layer, to designate a layer with a highest score representative of a best semantics in each test sample of a set of test sample processed;
extending, via the processor of the vehicle, a designated layer of the plurality of layers by a category branch to the NN to extract semantic data samples of the semantic content, and for an explainable classifier in the extension of the NN for defining a plurality of semantic categories determined by the NN;
training, via the processor of the vehicle, a set of connections of the explainable classifier of the NN to compute a set of output explanations with an accuracy measure associated with each output explanation based on at least one semantic category of the plurality of semantic categories;
comparing the accuracy measure for each output explanation based on an extracted semantic data sample by the trained explainable classifier for each semantic category to generate an output explanation in a user understandable format;
automatically controlling movement of the vehicle, via the processor, based on the input data and the neural network; and
providing, via the processor, an output for a user of the vehicle that includes an explanation of the controlling of the movement of the vehicle, using the output explanation.