US 12,443,726 B2
Measuring confounding effects in machine learning-based vulnerability discovery
Erik Imgrund, Heidelberg (DE); Tom Ganz, Karlsruhe (DE); and Martin Haerterich, Walldorf (DE)
Assigned to SAP SE, Walldorf (DE)
Filed by SAP SE, Walldorf (DE)
Filed on Nov. 29, 2023, as Appl. No. 18/522,389.
Prior Publication US 2025/0173442 A1, May 29, 2025
Int. Cl. G06N 7/01 (2023.01); G06F 21/57 (2013.01); G06N 20/00 (2019.01)
CPC G06F 21/577 (2013.01) [G06N 7/01 (2023.01); G06N 20/00 (2019.01); G06F 2221/033 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
a memory storing processor-executable program code; and
at least one processing unit to execute the processor-executable program code to cause the system to:
receive a plurality of code samples and corresponding ground truth classifications;
for each of the plurality of code samples, generate a plurality of perturbed code samples;
generate first probabilities for each of the plurality of code samples and the perturbed code samples using a first classification model;
generate second probabilities for each of the plurality of code samples and the perturbed code samples using a second model;
determine a causal probability for each code sample based on the first probabilities and the second probabilities;
determine a causal performance metric of the first classification model based on the ground truth classifications and the causal probabilities; and
present the causal performance metric.