	US 12,229,515 B2
	Adaptive knowledge graph representation learning method for integrating graph structure with text information
Huajun Chen, Zhejiang Province (CN); Yushan Zhu, Zhejiang Province (CN); and Wen Zhang, Zhejiang Province (CN)
Assigned to ZHEJIANG UNIVERSITY, Hangzhou (CN)
Appl. No. 17/923,566
Filed by ZHEJIANG UNIVERSITY, Zhejiang Province (CN)
PCT Filed Dec. 3, 2021, PCT No. PCT/CN2021/135512 § 371(c)(1), (2) Date Nov. 4, 2022, PCT Pub. No. WO2022/135120, PCT Pub. Date Jun. 30, 2022.
Claims priority of application No. 202011520309.5 (CN), filed on Dec. 21, 2020.
Prior Publication US 2023/0186030 A1, Jun. 15, 2023
Int. Cl. G06F 40/30 (2020.01); G06F 40/279 (2020.01); G06N 5/022 (2023.01)

CPC G06F 40/30 (2020.01) [G06F 40/279 (2020.01); G06N 5/022 (2013.01)]

4 Claims

1. An adaptive knowledge graph representation learning method for integrating a graph structure with text information, comprising the following steps:

(1) sampling a neighbor triple of each of a head entity and a tail entity in a target triple;

(2) calculating, by a semantic representation module (SRM), semantic representations of the target triple, and neighbor triples of its head and tail entities, wherein the semantic representation module (SRM) uses a pre-trained language model bert, and deletes a last classification layer of the model bert;

the semantic representation module inputs a sentence sequence formed by text descriptions about the head entity, a relation, and the tail entity of the triple in order, the text descriptions about the entity and the relation are separated by a delimiter [SEP], and an output identifier [OUT] is added to a header of the sequence;

the semantic representation module outputs an output vector corresponding to [OUT] in a last hidden layer of the module, and the output vector is the semantic representation of the correspondingly inputted triple;

the semantic representation of the target triple x is expressed by using the following formula:

q_x=SRM(x);

the semantic representations of neighbor triples of a head entities and a tail entities of the target triple x are expressed respectively by using the following formulas:

q_hⁱ=SRM(x_hⁱ),i=1,2, . . . a and

q_t^j=SRM(x_t^j),j=a+1,a+2, . . . 2a; and

each trained sample obtains one own semantic representation, semantic representations of a head-entity neighbor triples, and semantic representations of a tail-entity neighbor triples;

(3) inputting the semantic representations of the target triple, and the neighbor triples of its head and tail entities into a structure extraction module (SEM), and calculating structure representations of the head and tail entities of the target triple, wherein the structure extraction module (SEM) uses two layers of set-transformers; and

a specific process of calculating the structure representations of the head and tail entities of the target triple comprises:

arranging the semantic representations q_hⁱ(i=1, 2, . . . a) obtained in step (2) of the head-entity neighbor triples into a sequence, inputting the sequence into the structure extraction module (SEM), and taking an output of the structure extraction module as the structure representation of an entity h by using the following formula:

q_h*=SEM(q_h¹,q_h², . . . ,q_h^a), and

inputting the semantic representations of the α neighbor triples of the tail entity t of the target triple into the structure extraction module (SEM), and calculating the structure representation of the tail entity t by using the following formula:

q_t*=SEM(q_t^a+1,q_t^a+2, . . . ,q_t^2a);

(4) splicing the semantic representation of the target triple with the structure representations of its head and tail entities, inputting a spliced result into an adaptive classification layer, and calculating a loss by using an outputted probability distribution and a true label; and

(5) optimizing the foregoing module based on an optimization algorithm of gradient descent, until a loss value converges, to obtain a final spliced result between the semantic representation of the target triple and the structure representations of its head and tail entities.