US 12,314,861 B2
Systems and methods for semi-supervised learning with contrastive graph regularization
Junnan Li, Singapore (SG); and Chu Hong Hoi, Singapore (SG)
Assigned to Salesforce, Inc., San Francisco, CA (US)
Filed by Salesforce, Inc., San Francisco, CA (US)
Filed on Jan. 28, 2021, as Appl. No. 17/160,896.
Claims priority of provisional application 63/113,339, filed on Nov. 13, 2020.
Prior Publication US 2022/0156591 A1, May 19, 2022
Int. Cl. G06N 3/084 (2023.01); G06F 18/2137 (2023.01); G06F 18/214 (2023.01); G06F 18/22 (2023.01); G06F 18/2415 (2023.01)
CPC G06N 3/084 (2013.01) [G06F 18/21375 (2023.01); G06F 18/2155 (2023.01); G06F 18/22 (2023.01); G06F 18/2415 (2023.01)] 14 Claims
OG exemplary drawing
 
1. A method for semi-supervised learning with contrastive graph regularization, the method comprising:
receiving a batch of unlabeled image samples;
generating, from an unlabeled image sample, a weakly augmented image sample, a first strongly augmented image sample, and a second strongly augmented image sample;
generating, by an encoder and a classification of a neural model, a pseudo-label corresponding to the weakly augmented image sample, wherein the pseudo-label is generated by:
generating, by the encoder and the classification of the neural model, a classification probability for the weakly augmented image sample,
generating, by the encoder and a projection head of the neural model, an embedding for the weakly augmented image sample,
storing, at a memory bank, the generated classification probability and the generated embedding, and
aggregating class probabilities from neighboring samples in the memory bank to compute the pseudo-label;
generating, by the neural model, a first embedding corresponding to the first strongly augmented image sample and a second embedding corresponding to the second strongly augmented image sample;
building an embedding graph by comparing pairwise similarity between the first embedding and the second embedding;
generating a pseudo-label graph, comprising: constructing a similarity matrix among generated pseudo-labels corresponding to the batch of unlabeled image samples, wherein the similarity matrix contains a first similarity comparing each generated pseudo-label to itself, and a second similarity comparing each generated pseudo-label to another pseudo-label;
computing a contrastive loss based on a cross-entropy between the embedding graph and the pseudo-label graph;
computing an unsupervised classification loss based on a cross-entropy between the pseudo label and the classification probability for the weakly augmented image sample;
updating the neural model based at least in part on the contrastive loss and the unsupervised classification loss via backpropagation;
receiving, via a user data interface, an input image for classification; and
generating, via the neural model, an output classification for the input image.