US 12,243,294 B1
Image fingerprinting conversion between different image fingerprinting models
Jason B. Bryslawskyj, San Diego, CA (US); Yi Zhang, Santa Clara, CA (US); Ari Azarafrooz, Rancho Santa Margarita, CA (US); Wayne Xin, Santa Clara, CA (US); Yihua Liao, Palo Alto, CA (US); Niranjan Koduri, Pleasanton, CA (US); and Emanoel Daryoush, San Jose, CA (US)
Assigned to Netskope, Inc., Santa Clara, CA (US)
Filed by Netskope, Inc., Santa Clara, CA (US)
Filed on Aug. 16, 2023, as Appl. No. 18/451,014.
Int. Cl. G06V 10/764 (2022.01); G06V 10/74 (2022.01); G06V 10/762 (2022.01); G06V 10/774 (2022.01)
CPC G06V 10/774 (2022.01) [G06V 10/761 (2022.01); G06V 10/762 (2022.01); G06V 10/764 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method, comprising:
generating a plurality of first image fingerprint embeddings, the generating comprising:
providing each of a plurality of images as input to a first image fingerprinting model, wherein the first image fingerprinting model is trained to generate a first image fingerprint embedding comprising a multi-dimensional feature vector representing visual aspects of the input image, and
receiving each of the plurality of first image fingerprint embeddings as output from the first image fingerprinting model;
generating a plurality of second image fingerprint embeddings, the generating comprising:
providing each of the plurality of images as input to a second image fingerprinting model, wherein the second image fingerprinting model is trained to generate a second image fingerprint embedding comprising a multi-dimensional feature vector representing visual aspects of the input image, and
receiving each of the plurality of second image fingerprint embeddings as output from the second image fingerprinting model, wherein the first image fingerprint embedding is different than the second image fingerprint embedding generated for each of the plurality of images;
training a converter model to convert a first image fingerprint embedding to a second image fingerprint embedding compatible with other second image fingerprint embeddings, the training comprising:
for each image of the plurality of images:
providing the first image fingerprint embedding associated with the respective image as input to the converter model,
receiving an output image fingerprint embedding from the converter model,
comparing the output image fingerprint embedding with the second image fingerprint embedding associated with the respective image to calculate a loss, and
backpropagating the loss through the converter model;
accessing a plurality of stored first image fingerprint embeddings;
providing each of the stored first image fingerprint embeddings as input to the converter model to generate a second plurality of second image fingerprint embeddings; and
adding the second plurality of second image fingerprint embeddings to an index comprising a third plurality of second image fingerprint embeddings generated by the second image fingerprinting model, wherein the index is used by an approximate nearest neighbors model.