US 12,462,157 B2
Automatic channel pruning via graph neural network based hypernetwork
Baopu Li, Santa Clara, CA (US); Qiuling Suo, Sunnyvale, CA (US); and Yuchen Bian, Santa Clara, CA (US)
Assigned to Baidu USA LLC, Sunnyvale, CA (US)
Filed by Baidu USA, LLC, Sunnyvale, CA (US)
Filed on Jun. 22, 2022, as Appl. No. 17/846,555.
Claims priority of provisional application 63/241,095, filed on Sep. 6, 2021.
Prior Publication US 2023/0084203 A1, Mar. 16, 2023
Int. Cl. G06N 3/082 (2023.01); G06N 3/04 (2023.01); G06N 3/0499 (2023.01); G06N 3/0985 (2023.01); G06N 3/0455 (2023.01)
CPC G06N 3/082 (2013.01) [G06N 3/04 (2013.01); G06N 3/0499 (2023.01); G06N 3/0985 (2023.01); G06N 3/0455 (2023.01)] 15 Claims
OG exemplary drawing
 
1. A computer-implemented method for using a neural network pruned via a graph neural network based hypernetwork, the method comprising:
providing input data to a final pruned neural network, the final pruned neural network being pruned via a graph neural network based hypernetwork pruning process, the graph neural network based hypernetwork pruning process comprising:
obtaining a pretrained neural network;
constructing an information flow graph based upon the pretrained neural network, the information flow graph comprising a node set and an edge set;
initializing node embeddings for nodes of the information flow graph;
responsive to a stop condition not being satisfied, performing steps associated with a graph neural network based hypernetwork, the steps comprising:
updating the node embeddings by applying a graph neural network to the information flow graph;
determining importance vectors of the pretrained neural network by applying a transformation function to at least some of the updated node embeddings;
determining mask indicators for channels of the pretrained neural network by applying normalization and/or binarization operations to the importance vectors, the mask indicators indicating which of the channels of the pretrained neural network to prune to obtain a pruned neural network;
determining weights for the pruned neural network by applying a feed-forward neural network to a function of the updated embeddings and corresponding importance vectors for the pruned neural network;
calculating a loss using a training data set; and
updating hypernetwork weights for the graph neural network based hypernetwork;
responsive to the stop condition being satisfied, providing one or more output pruned neural networks; and
selecting the final pruned neural network from the one or more output pruned neural networks; and
obtaining one or more output labels generated by the final pruned neural network in response to the input data.