US 12,106,217 B2
Graph neutral networks with attention
Paidi Creed, London (GB); Aaron Sim, London (GB); Amir Alamdari, London (GB); Joss Briody, London (GB); Daniel Neil, Williamsburg, VA (US); and Alix Lacoste, Brooklyn, NY (US)
Assigned to BenevolentAI Technology Limited, London (GB)
Appl. No. 17/041,625
Filed by BENEVOLENTAI TECHNOLOGY LIMITED, London (GB)
PCT Filed May 16, 2019, PCT No. PCT/GB2019/051352
§ 371(c)(1), (2) Date Sep. 25, 2020,
PCT Pub. No. WO2019/220128, PCT Pub. Date Sep. 21, 2019.
Claims priority of provisional application 62/673,554, filed on May 18, 2018.
Prior Publication US 2021/0081717 A1, Mar. 18, 2021
Int. Cl. G06K 9/00 (2022.01); G06F 17/16 (2006.01); G06F 18/214 (2023.01); G06N 3/047 (2023.01); G06N 3/08 (2023.01); G06N 3/082 (2023.01); G06N 5/02 (2023.01)
CPC G06N 3/082 (2013.01) [G06F 17/16 (2013.01); G06F 18/2148 (2023.01); G06N 3/047 (2023.01); G06N 3/08 (2013.01); G06N 5/02 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A computer-implemented method for training a graph neural network (GNN) model based on an entity-entity graph for relationship prediction, the entity-entity graph comprising a plurality of entity nodes in which each entity node is connected to one or more entity nodes of the plurality of entity nodes by one or more corresponding relationship edges, the method comprising:
generating an embedding based on data representative of at least a portion of the entity-entity graph for the GNN model, wherein the embedding comprises an attention weight assigned to each relationship edge of at least the portion of the entity-entity graph; and
updating the GNN model including the attention weights by minimising a loss function custom character(⋅) associated with at least the embedding;
wherein the attention weights indicate the relevancy of each corresponding relationship edge between entity nodes of the entity-entity graph and assist the GNN model in relationship prediction,
wherein minimising the loss function, custom character(∩), comprises:
assigning an attention weight, αr,i,j, to each relationship edge of the entity-entity graph, wherein αr,i,j, is the attention weight associated with the connection between entity node i and entity node j with relation index r of the plurality of relationships associated with the entity-entity graph;
modifying the loss function by adding an attention loss function, custom characteratt(Λ) associated with the set of attention weights Λ={αr,i,j} to the loss function; and
minimising the modified loss function, custom charactermod, with respect to the weights of the GNN model including the attention weights;
wherein the modified loss function is custom charactermod(⋅)=custom character(⋅)+custom characteratt(Λ).