US 11,947,503 B2
Autoregressive graph generation machine learning models
Hanjun Dai, Atlanta, GA (US); Azade Nazi, San Jose, CA (US); Yujia Li, London (GB); Bo Dai, San Jose, CA (US); and Dale Eric Schuurmans, Edmonton (CA)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Jun. 17, 2021, as Appl. No. 17/351,086.
Prior Publication US 2022/0414067 A1, Dec. 29, 2022
Int. Cl. G06F 16/21 (2019.01); G06F 16/22 (2019.01); G06N 3/045 (2023.01)
CPC G06F 16/212 (2019.01) [G06F 16/2237 (2019.01); G06F 16/2246 (2019.01); G06N 3/045 (2023.01)] 26 Claims
OG exemplary drawing
 
1. A method performed by one or more computers, the method comprising:
generating data defining a graph using a neural network system with a set of neural network parameters having trained values that have been determined by a machine learning training technique, wherein the graph comprises: (i) a plurality of nodes, and (ii) a plurality of edges that each connect a respective pair of nodes in the graph, wherein generating the graph using the neural network system comprises:
determining a number of nodes in the graph;
sequentially generating a respective edge set for each node in the graph starting from a first node in an ordering of the nodes in the graph, wherein the edge set for each node defines a set of edges in the graph corresponding to the node, wherein for each of a plurality of nodes after a first node in the ordering of nodes, generating the edge set for the node comprises:
receiving a context embedding for the node that summarizes a respective edge set for each node that precedes the node in the ordering of the nodes;
generating, based on the context embedding for the node: (i) a respective edge set for the node, and (ii) a respective embedding of the edge set for the node, using the neural network system and in accordance with the trained values of the set of neural network parameters of the neural network system;
generating a context embedding for a next node in the ordering of the nodes using the embedding of the edge set for the node using the neural network system and in accordance with the trained values of the set of neural network parameters of the neural network system, comprising:
updating a first level of a hierarchical arrangement of embeddings having a plurality of levels by adding the embedding of the edge set for the node to the first level;
propagating the update to the first level in the hierarchical arrangement of embeddings into each subsequent level in the hierarchical arrangement of embeddings,
wherein each embedding in each level after the first level in the hierarchical arrangement of embeddings is derived from two or more embeddings at a preceding level in the hierarchical arrangement of embeddings;
generating the context embedding for the next node based on the updated hierarchical arrangement of embeddings; and
providing the updated hierarchical arrangement of embeddings for use in generating the edge set for a next node in the ordering of the nodes; and
adding the set of edges defined by the edge set for the node to the graph; and
providing an output comprising the data defining the graph.