US 12,236,326 B2
Attribution and generation of saliency visualizations for machine-learning models
Andrei Kapishnikov, Watertown, MA (US); Fernanda Bertini Viégas, Lexington, MA (US); Michael Andrew Terry, Cambridge, MA (US); and Tolga Bolukbasi, Cambridge, MA (US)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Aug. 1, 2023, as Appl. No. 18/363,277.
Application 18/363,277 is a continuation of application No. 16/719,244, filed on Dec. 18, 2019, granted, now 11,755,948.
Prior Publication US 2024/0054402 A1, Feb. 15, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06N 20/00 (2019.01); G06N 5/04 (2023.01); G06N 20/10 (2019.01)
CPC G06N 20/00 (2019.01) [G06N 5/04 (2013.01); G06N 20/10 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method, comprising:
receiving, by a computing system comprising one or more processors, an input comprising a plurality of features;
segmenting, by the computing system, the input into a plurality of regions, at least some regions of the plurality of regions overlapping with one or more other regions of the plurality of regions;
determining, by the computing system, attribution scores for each of the plurality of features;
determining, by the computing system, respective gain values for each respective region of the plurality of regions based on a sum of the attribution scores for features among the plurality of features within the respective region divided by an area associated with the respective region;
based on a first gain value for a first region among the plurality of regions being greater than a second gain value for a second region among the plurality of regions, selecting, by the computing system, the first region to add to a saliency mask; and
outputting, by the computing system, the saliency mask.