US 11,783,190 B2
Method and device for ascertaining an explanation map
Joerg Wagner, Renningen (DE); Tobias Gindele, Zürich (CH); Jan Mathias Koehler, Stuttgart (DE); Jakob Thaddaeus Wiedemer, Laichingen (DE); and Leon Hetzel, Werther (DE)
Assigned to ROBERT BOSCH GMBH, Stuttgart (DE)
Appl. No. 17/261,810
Filed by Robert Bosch GmbH, Stuttgart (DE)
PCT Filed Jul. 3, 2019, PCT No. PCT/EP2019/067840
§ 371(c)(1), (2) Date Jan. 20, 2021,
PCT Pub. No. WO2020/025244, PCT Pub. Date Feb. 6, 2020.
Claims priority of application No. 102018213052.3 (DE), filed on Aug. 3, 2018.
Prior Publication US 2021/0279529 A1, Sep. 9, 2021
Int. Cl. G06N 3/08 (2023.01); G06N 3/084 (2023.01); G06N 3/04 (2023.01); G06F 18/241 (2023.01); G06F 18/40 (2023.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 10/44 (2022.01)
CPC G06N 3/084 (2013.01) [G06F 18/241 (2023.01); G06F 18/41 (2023.01); G06N 3/04 (2013.01); G06N 3/08 (2013.01); G06V 10/454 (2022.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 2201/03 (2022.01)] 24 Claims
OG exemplary drawing
 
1. A method comprising:
ascertaining an explanation map of an image indicating an extent to which different parts of an input image, which is classified by a deep neural network (a) into which the input image is input and (b) that includes a plurality of layers of neurons that are each activatable, affected the classification made by the deep neural network, wherein the input image is formed of a plurality of pixels, the ascertaining being performed by:
identifying a subset of the plurality of pixels as being significant for the classification of the image ascertained using a deep neural network; and
generating, as the explanation map, a modified version of the image in which the identified subset of the plurality of pixels are highlighted;
wherein the identifying of the subset of the plurality of pixels is performed:
in a manner by which input of the explanation map into the deep neural network results in a classification by the deep neural network that is the same as the classification, by the deep neural network, of the input image; and
by optimizing a lost function, the optimization of the lost function ensuring no exceedance of activations of the neurons of respective ones of the plurality of layers caused by the input of the explanation map into the deep neural network over activations of the neurons of the respective layer caused by the input of the input image into the deep neural network.