US 12,293,294 B2
Systems and methods for image or video performance heat map generation
Elham Saraee, Jamaica Plain, MA (US); Jehan Hamedi, South Boston, MA (US); Zachary Halloran, Braintree, MA (US); and Arsenii Mustafin, Boston, MA (US)
Assigned to VIZIT LABS, INC., Boston, MA (US)
Filed by Vizit Labs, Inc., Boston, MA (US)
Filed on May 10, 2021, as Appl. No. 17/316,503.
Application 17/316,503 is a continuation in part of application No. 16/236,298, filed on Dec. 28, 2018, granted, now 11,301,718.
Prior Publication US 2021/0264161 A1, Aug. 26, 2021
Int. Cl. G06N 3/084 (2023.01); G06N 3/08 (2023.01); G06Q 30/0241 (2023.01); G06T 11/00 (2006.01); G06T 11/60 (2006.01); G06V 10/764 (2022.01); G06V 10/776 (2022.01); G06V 20/40 (2022.01)
CPC G06N 3/084 (2013.01) [G06N 3/08 (2013.01); G06Q 30/0277 (2013.01); G06T 11/001 (2013.01); G06T 11/60 (2013.01); G06V 10/764 (2022.01); G06V 10/776 (2022.01); G06V 20/46 (2022.01)] 17 Claims
OG exemplary drawing
 
1. A method for improved explainability of a neural network, comprising:
receiving, by one or more processors, an image or video comprising a set of pixels;
extracting, by the one or more processors, a plurality of features from the image or video;
generating, by the one or more processors, a feature vector including the plurality of features each at different index values of the feature vector, each of the plurality of features corresponding to a different pixel or subset of pixels of the set of pixels of the image or video;
executing, by the one or more processors, a neural network using the plurality of features to obtain a performance score for the image or video, the neural network comprising an input layer, a plurality of intermediate layers subsequent to the input layer, and a regression layer or a classification layer subsequent to the plurality of intermediate layers;
extracting, by the one or more processors, one or more values from one or more signals between an intermediate layer of the plurality of intermediate layers and the regression layer or the classification layer, the one or more values generated responsive to executing the neural network using the extracted plurality of features;
back-propagating, by the one or more processors, a set of the extracted one or more values through the plurality of intermediate layers and the input layer;
extracting, by the one or more processors, an impact score for each of the plurality of features from the input layer and generated from the back-propagation of the set of the extracted one or more values through the plurality of intermediate layers and the input layer; and
assigning, by the one or more processors, an impact score of the extracted impact scores to each pixel and/or subset of pixels of the set of pixels of the image or video based on the pixel or subset of pixels mapping to the same index value of the feature vector as the impact score;
determining, by the one or more processors, (i) a first plurality of pixels of the image or video that correspond to a first color indicating a positive impact to the performance score, and (ii) a second plurality of pixels of the image or video that correspond to a second color indicating a negative impact to the performance score, based on an assignment of a positive impact score to each of the first plurality of pixels and a negative impact score to each of the second plurality of pixels;
generating, by the one or more processors, a heat map overlay comprising the first color at locations of the image or video corresponding to the first plurality of pixels and the second color at locations of the image or video corresponding to the second plurality of pixels; and
presenting, at a display of a client device and by the one or more processors, the heat map overlay over the image or video.