US 12,135,701 B2
Self contrastive decorrelation based training of machine learning models
Tassilo Klein, Berlin (DE); and Moin Nabi, Berlin (DE)
Assigned to SAP SE, Walldorf (DE)
Filed by SAP SE, Walldorf (DE)
Filed on Dec. 15, 2022, as Appl. No. 18/081,920.
Prior Publication US 2024/0202176 A1, Jun. 20, 2024
Int. Cl. G06F 16/00 (2019.01); G06F 16/22 (2019.01); G06F 16/242 (2019.01); G06F 17/15 (2006.01)
CPC G06F 16/2237 (2019.01) [G06F 16/243 (2019.01); G06F 17/15 (2013.01)] 15 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
training a machine learning model, the training comprising:
receiving a sentence including a plurality of text;
performing a first encoding operation on the sentence, the first encoding operation comprising generating a first vector representation of the sentence, the first vector representation including first numerical elements representing the plurality of text of the sentence, wherein the first vector representation is generated using a first dropout ratio that modifies, in a vector domain, the sentence by masking one or more of the first numerical elements of the first vector representation;
performing a second encoding operation on the sentence, the second encoding operation comprising generating a second vector representation of the sentence, the second vector representation including second numerical elements representing the plurality of text of the sentence, wherein the second vector representation is generated using a second dropout ratio that modifies, in the vector domain, the sentence by masking one or more of the second numerical elements of the second vector representation, wherein the second dropout ratio is larger than the first dropout ratio;
mapping the first vector representation to a first high dimensional vector representation, and mapping the second vector representation to a second high dimensional vector representation;
generating a correlation matrix using the first high dimensional vector representation and the second high dimensional vector representation; and
performing a decorrelation operation on the correlation matrix to decorrelate the first high dimensional vector representation and the second high dimensional vector representation, a trained machine learning model comprising the decorrelated first high dimensional vector representation and the decorrelated second high dimensional vector representation;
receiving, by the trained machine learning model, a query that includes a target sentence; and
outputting, using the trained machine learning model, a result sentence that satisfies a similarity metric relative to the target sentence.