US 12,271,447 B2
Methods and servers for determining metric-specific thresholds to be used with a plurality of nested metrics for binary classification of a digital object
Aleksey Vasilevich Toshchakov, Vologda (RU); Mikhail Mikhailovich Nosovsky, Moscow (RU); and Artem Vladimirovich Meshcheryakov, Shchelkovo (RU)
Assigned to Y.E. Hub Armenia LLC, Yerevan (AM)
Filed by YANDEX EUROPE AG, Lucerne (CH)
Filed on Oct. 5, 2021, as Appl. No. 17/494,405.
Claims priority of application No. RU2020133324 (RU), filed on Oct. 9, 2020.
Prior Publication US 2022/0114402 A1, Apr. 14, 2022
Int. Cl. G06F 18/241 (2023.01); G06F 18/21 (2023.01); G06F 18/2431 (2023.01); G06N 20/00 (2019.01)
CPC G06F 18/241 (2023.01) [G06F 18/217 (2023.01); G06F 18/2431 (2023.01); G06N 20/00 (2019.01)] 18 Claims
OG exemplary drawing
 
1. A method of determining a target combination of metric-specific thresholds to be used with a plurality of nested metrics for performing binary classification of a digital object into a first class or a second class, the object being associated with past object events an indication of which is stored in a storage, the method executable by a server configured to access the storage, the method comprising:
acquiring, by the server, a plurality of object-specific validation datasets, a given one of the plurality of object-specific validation datasets comprising an indication of a plurality of past object events associated with a respective validation object and a ground-truth class of the respective validation object being one of the first class and the second class;
applying, by the server, a plurality of nested metrics onto the plurality of object-specific validation datasets, thereby generating a plurality of prediction values,
a given prediction value being indicative of a respective probability of the respective validation object belonging to one of the first class and the second class;
during a first iteration:
comparing, by the server, the plurality of predictions values against respective ones from a first combination of metric-specific thresholds for determining predicted classes of the respective validation objects for the first iteration;
generating, by the server, first precision parameters and first recall parameters for the plurality of nested metrics for the first iteration by comparing the ground-truth classes against the respective predicted classes of respective validation objects of the first iteration;
during a second iteration:
adjusting, by the server, one of the first combination of metric-specific thresholds thereby generating a second combination of metric-specific thresholds;
comparing, by the server, the plurality of predictions values against respective ones from the second combination of metric-specific thresholds for determining predicted classes of the respective validation objects for the second iteration;
generating, by the server, second precision parameters and second recall parameters for the plurality of nested metrics for the second iteration by comparing the ground-truth classes against the respective predicted classes of respective validation objects of the second iteration; and
selecting, by the server, one of the first combination of metric-specific thresholds and the second combination of metric-specific thresholds as the target combination of metric-specific thresholds by:
comparing at least one of (i) the first precision parameters and the second precision parameters against a precision threshold, and (ii) the first recall parameters and the second recall parameters against a recall threshold, and
the target combination of metric-specific thresholds to be used with the plurality of nested metrics in an in-use mode for performing binary classification of the digital object,
such that in response to an in-use predicted value of at least one of the plurality of nested metrics for the digital object being above a respective one of the target combination of metric-specific thresholds, determining the digital object to be of the first class.