US 12,072,974 B2
Detecting an algorithmic attack against a hosted AI system based on inputs and outputs of the hosted AI system
Hyrum Spencer Anderson, Eagle, ID (US); Raja Sekhar Rao Dheekonda, Bellevue, WA (US); William Pearce, Highland, UT (US); Ricky Dee Loynd, Redmond, WA (US); James David McCaffrey, Issaquah, WA (US); and Ram Shankar Siva Kumar, Bothell, WA (US)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Apr. 6, 2022, as Appl. No. 17/715,014.
Prior Publication US 2023/0325495 A1, Oct. 12, 2023
Int. Cl. G06F 21/55 (2013.01); G06N 5/02 (2023.01)
CPC G06F 21/554 (2013.01) [G06N 5/02 (2013.01); G06F 2221/034 (2013.01)] 23 Claims
OG exemplary drawing
 
1. A system comprising:
a memory; and
a processing system coupled to the memory, the processing system configured to:
derive features, which are associated with a known type of algorithmic attack, from numerical representations of queries that are received by a hosted artificial intelligence system and outputs that result from processing of the queries by the hosted artificial intelligence system;
use a feature-based classifier model to generate a classification score, which indicates a likelihood that at least a portion of the queries corresponds to the known type of algorithmic attack, by providing the derived features as inputs to the feature-based classifier model;
compare the classification score to a score threshold that is associated with the known type of algorithmic attack; and
detect an algorithmic attack based at least on the classification score being greater than or equal to the score threshold that is associated with the known type of algorithmic attack.