US 12,443,827 B2
System, method, and computer program product for saving memory during training of knowledge graph neural networks
Huiyuan Chen, San Jose, CA (US); Xiaoting Li, Sunnyvale, CA (US); Michael Yeh, Newark, CA (US); Yan Zheng, Los Gatos, CA (US); and Hao Yang, San Jose, CA (US)
Assigned to Visa International Service Association, San Francisco, CA (US)
Appl. No. 18/844,254
Filed by Visa International Service Association, San Francisco, CA (US)
PCT Filed May 1, 2023, PCT No. PCT/US2023/020540
§ 371(c)(1), (2) Date Sep. 5, 2024,
PCT Pub. No. WO2023/215214, PCT Pub. Date Nov. 9, 2023.
Claims priority of provisional application 63/337,329, filed on May 2, 2022.
Prior Publication US 2025/0111213 A1, Apr. 3, 2025
Int. Cl. G06N 3/0495 (2023.01); G06N 3/084 (2023.01)
CPC G06N 3/0495 (2023.01) [G06N 3/084 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A computer-implemented method, comprising:
receiving, with at least one processor, a training dataset comprising a first set of knowledge graph embeddings associated with a plurality of entities for a first layer of a knowledge graph;
inputting, with the at least one processor, the training dataset into a knowledge graph neural network;
training, with the at least one processor, based on the training dataset, the knowledge graph neural network to:
train parameters of the knowledge graph neural network; and
generate at least one further set of knowledge graph embeddings associated with the plurality of entities for at least one further layer of the knowledge graph, wherein each further set of knowledge graph embeddings of the at least one further set of knowledge graph embeddings comprises a respective activation map of the knowledge graph neural network for a respective layer of the at least one further layer;
quantizing, with the at least one processor, the at least one further set of knowledge graph embeddings to provide at least one set of quantized knowledge graph embeddings;
storing, with the at least one processor, the at least one set of quantized knowledge graph embeddings in a memory without storing the at least one further set of knowledge graph embeddings;
dequantizing, with the at least one processor, the at least one set of quantized knowledge graph embeddings to provide at least one set of dequantized knowledge graph embeddings, wherein the at least one set of dequantized knowledge graph embeddings approximates the at least one further set of knowledge graph embeddings;
determining, with the at least one processor, gradients for backpropagation based on the at least one set of dequantized knowledge graph embeddings; and
updating, with the at least one processor, the parameters of the knowledge graph neural network based on the gradients.