US 11,899,794 B1
Machine learning model robustness characterization
Neil Serebryany, Los Angeles, CA (US); Brendan Quinlivan, San Francisco, CA (US); Victor Ardulov, Long Beach, CA (US); Ilja Moisejevs, Dublin (IE); and David Richard Gibian, New York, NY (US)
Assigned to CALYPSO AI CORP, San Mateo, CA (US)
Filed by CALYPSO AI CORP, San Mateo, CA (US)
Filed on Oct. 21, 2020, as Appl. No. 17/076,675.
Application 17/076,675 is a continuation of application No. 16/788,200, filed on Feb. 11, 2020, granted, now 10,846,407.
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 21/56 (2013.01); G06N 20/00 (2019.01)
CPC G06F 21/566 (2013.01) [G06N 20/00 (2019.01); G06F 2221/033 (2013.01)] 5 Claims
OG exemplary drawing
 
1. A system for characterizing a robustness of a machine learning model comprising:
at least one data processor; and
memory including instructions, which when executed by the at least one data processor, result in operations comprising:
receiving a file with a known, first classification by the machine learning model;
automatically selecting which of a plurality of perturbation algorithms to use to modify the file, the perturbation algorithm being selected as to provide a shortest sequence of actions to cause the machine learning model to provide a desired classification; and
iteratively modifying the received file using the selected perturbation algorithm and inputting the corresponding modified file into the machine learning model until the machine learning model outputs a known, second classification.