US 11,934,634 B2
System, user interface and method for interactive negative explanation of machine learning localization models in health care applications
Marcin Sieniek, Mountain View, CA (US)
Assigned to Google LLC, Mountain View, CA (US)
Appl. No. 17/422,356
Filed by GOOGLE LLC, Monntain View, CA (US)
PCT Filed Oct. 10, 2019, PCT No. PCT/US2019/055629
§ 371(c)(1), (2) Date Jul. 12, 2021,
PCT Pub. No. WO2020/146024, PCT Pub. Date Jul. 16, 2020.
Application 17/422,356 is a division of application No. 16/246,156, filed on Jan. 11, 2019, granted, now 10,936,160.
Prior Publication US 2022/0121330 A1, Apr. 21, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 3/0482 (2013.01); G06F 3/0354 (2013.01); G06F 3/04883 (2022.01); G06N 3/08 (2023.01); G06N 5/04 (2023.01); G06T 7/00 (2017.01); G16H 50/20 (2018.01); G16H 50/30 (2018.01)
CPC G06F 3/0482 (2013.01) [G06F 3/03543 (2013.01); G06F 3/04883 (2013.01); G06N 3/08 (2013.01); G06N 5/04 (2013.01); G06T 7/0012 (2013.01); G16H 50/20 (2018.01); G16H 50/30 (2018.01); G06T 2207/30096 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A method comprising the steps of:
a) presenting an image with an overall risk score or classification produced, based on medical image data related to the image, by a machine learning model as to a disease state of a patient represented by the image, wherein the image is augmented with highlighting to indicate two or more regions in the image which were detected by a detection model of the machine learning model and which were, responsive to detection by the detection model, subjected to inference by the machine learning model to generate respective first region-level inferences that were then used to generate the overall risk score or classification produced by the machine learning model;
b) receiving, via a user interface tool, a user input highlighting two or more regions of the image;
c) subjecting the highlighted two or more regions to inference by the machine learning model to generate respective second region-level inferences therefor and generating an updated overall risk score or classification based on the first region-level inferences and the second region-level inferences, wherein generating the updated overall risk score or classification based on the first region-level inferences and the second region-level inferences comprises generating an updated overall risk that is increased by positive values among the second region-level inferences and decreased by increased areas represented by the user-highlighted two or more regions; and
d) presenting the updated overall risk score or classification to the user via a display.