US 12,282,485 B2
Word embedding quality assessment through asymmetry
Wei Zhang, Acton, MA (US); Yang Yu, Acton, MA (US); Murray Scott Campbell, Yorktown Heights, NY (US); and Sadhana Kumaravel, White Plains, NY (US)
Assigned to International Business Machines Corporation, Armonk, NY (US)
Filed by INTERNATIONAL BUSINESS MACHINES CORPORATION, Armonk, NY (US)
Filed on Aug. 28, 2020, as Appl. No. 17/005,471.
Prior Publication US 2022/0067051 A1, Mar. 3, 2022
Int. Cl. G06F 40/237 (2020.01); G06F 16/2457 (2019.01); G06F 40/30 (2020.01); G06N 3/02 (2006.01)
CPC G06F 16/24578 (2019.01) [G06F 40/237 (2020.01); G06N 3/02 (2013.01); G06F 40/30 (2020.01)] 6 Claims
OG exemplary drawing
 
1. A computer-implemented method to determine the quality of word embeddings assigned by a word embedding model, the computer-implemented method comprising:
processing, by one or more processors, a corpus, based on a natural language preprocessing system, wherein processing comprises one or more of the following techniques: tokenization, part-of-speech tagging, semantic relationship identification, or syntactic relationship identification;
generating, by one or more processors, a word embedding for each word of a plurality of words in the processed corpus, based at least in part on a word embedding model with transformer neural network architecture, wherein generating the word embedding for each word of a plurality of words in the processed corpus further comprises assigning, by the one or more processors, a vector to each dimension within an n-dimensional space;
generating, by the one or more processors, a first asymmetry score for the embedding of a first word from the plurality of words in the processed corpus and the embedding of a second word from the plurality of words in the processed corpus by calculating a first log asymmetric ratio of the first word and the second word;
generating, by the one or more processors, a second asymmetry score that is a second log asymmetric ratio based, at least in part, on evocation data corresponding to the first word embedding and evocation data corresponding to the second word embedding wherein evocation data from evocation database;
comparing, by the one or more processors, the first asymmetry score to the second asymmetry score, based on one of the following, Kendall tau rank correlation coefficient, distance correlation, or polychoric correlation;
generating, by one or more processors, an embedding quality score that measures the quality of word representations of word embedding models through a degree of asymmetry between the first word and the second word based at least in part on the comparing of the first asymmetry score and the second asymmetry score.