US 12,271,429 B2
Quantifying and improving the performance of computation-based classifiers
Debraj Debashish Basu, Sunnyvale, CA (US); Ganesh Satish Mallya, Santa Clara, CA (US); Shankar Venkitachalam, Santa Clara, CA (US); and Deepak Pai, Sunnyvale, CA (US)
Assigned to Adobe Inc., San Jose, CA (US)
Filed by ADOBE INC., San Jose, CA (US)
Filed on Mar. 16, 2021, as Appl. No. 17/203,300.
Prior Publication US 2022/0300557 A1, Sep. 22, 2022
Int. Cl. G06F 16/906 (2019.01); G06F 16/901 (2019.01); G06F 16/9035 (2019.01); G06N 5/02 (2023.01)
CPC G06F 16/906 (2019.01) [G06F 16/9027 (2019.01); G06F 16/9035 (2019.01); G06N 5/02 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
accessing a set of datapoints, wherein a ground-truth label is assigned to each datapoint of the set of datapoints, a computation-based classifier model comprising a set of labels predicts a predicted label for each datapoint, and the ground-truth label and the predicted label for each datapoint is included in the set of labels;
selecting, for each label of the set of labels, a subset of the set of datapoints wherein the subset of the set of datapoints is randomly selected from datapoints associated with a corresponding label;
determining, for each label of the set of labels, a credibility interval using the randomly selected subset of the set of datapoints wherein the credibility interval is determined based on instances of correct and incorrect label predictions of the computation-based classifier model for the label using the randomly selected subset of the set of datapoints;
identifying a first label of the set of labels with a first credibility interval for a performance metric of the first label being greater than a predetermined interval threshold;
identifying a second label of the set of labels, based on instances of incorrect label predictions of the computation-based classifier model for each of the first label and the second label, wherein the instances of incorrect label predictions of the computation-based classifier model for each of the first label and the second label indicate that, when predicting labels for the set of datapoints, the computation-based classifier model is likely to confuse the first label with the second label; and
updating the computation-based classifier model based on a third label that includes an aggregation of the first label and the second label.