US 12,266,209 B1
Image classifier automated testing and outlier detection
Jason B. Bryslawskyj, San Diego, CA (US); Yi Zhang, Santa Clara, CA (US); Emanoel Daryoush, San Jose, CA (US); Ari Azarafrooz, Rancho Santa Margarita, CA (US); Wayne Xin, Santa Clara, CA (US); Yihua Liao, Palo Alto, CA (US); and Niranjan Koduri, Pleasanton, CA (US)
Assigned to Netskope, Inc., Santa Clara, CA (US)
Filed by Netskope, Inc., Santa Clara, CA (US)
Filed on Feb. 26, 2024, as Appl. No. 18/587,267.
Int. Cl. G06V 20/70 (2022.01); G06V 10/762 (2022.01); G06V 40/12 (2022.01)
CPC G06V 40/1365 (2022.01) [G06V 10/762 (2022.01); G06V 20/70 (2022.01); G06V 40/1347 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method of determining a quality an image classifier, the method comprising:
receiving a plurality of images provided by a user and a user-defined label for the plurality of images;
generating, with an image fingerprint model, a plurality of image fingerprint embeddings, wherein each image fingerprint embedding of the plurality of image fingerprint embeddings represents an associated image of the plurality of images;
creating the image classifier using each image fingerprint embedding in the plurality of image fingerprint embeddings and a plurality of negative image fingerprint embeddings;
creating a plurality of test classifiers, wherein the creating comprises creating one test classifier of the plurality of test classifiers for each image fingerprint embedding in the plurality of image fingerprint embeddings, wherein creating each test classifier of the plurality of test classifiers comprises:
selecting an image fingerprint embedding of the plurality of image fingerprint embeddings as a test embedding for which to create the respective test classifier, and
creating the respective test classifier using the plurality of negative image fingerprint embeddings and each of the plurality of image fingerprint embeddings except the selected image fingerprint embedding;
applying each of the plurality of test classifiers to its respective test embedding to obtain a plurality of recall results; and
assessing the quality of the image classifier based on the plurality of recall results.