US 12,306,745 B2
Method for assessing test adequacy of neural network based on element decomposition
Yinxiao Miao, Beijing (CN); Yifei Liu, Beijing (CN); Ping Yang, Beijing (CN); Xiujian Zhang, Beijing (CN); Zhonghao Cheng, Beijing (CN); Long Zhang, Beijing (CN); Tianqi Wan, Beijing (CN); Haoyi Chen, Beijing (CN); Jing Sun, Beijing (CN); and Yijia Ding, Beijing (CN)
Assigned to BEIJING AEROSPACE INSTITUTE FOR METROLOG AND MEASUREMENT TECHNOLOGY, Beijing (CN)
Filed by BEIJING AEROSPACE INSTITUTE FOR METROLOGY AND MEASUREMENT TECHNOLOGY, Beijing (CN)
Filed on Nov. 22, 2023, as Appl. No. 18/518,043.
Claims priority of application No. 202311041164.4 (CN), filed on Aug. 18, 2023.
Prior Publication US 2024/0095159 A1, Mar. 21, 2024
Int. Cl. G06F 11/3668 (2025.01)
CPC G06F 11/3688 (2013.01) 5 Claims
OG exemplary drawing
 
1. A method for assessing test adequacy of a neural network based on element decomposition, comprising the following steps:
(S1) dividing neural network testing into black box testing and white box testing; decomposing key elements of the black box testing and key elements of the white box testing; and defining a test adequacy of the key elements of the black box testing and a test adequacy of the key elements of the white box testing;
(S2) extracting network parameters of each layer in the neural network during testing; wherein the network parameters comprise a weight matrix and a bias vector, and test procedure parameters comprise maximum values and minimum values of neuron activation;
(S3) inversely calculating importance values of neurons in each layer of the neural network under each test case based on weight parameters and activation values extracted in step (S2); clustering the importance values of neurons in each layer; and generating a neuron importance value heat map of each layer based on clustering results;
(S4) mutating a model of the neural network according to the clustering results to obtain a mutated model, testing the mutated model and performing test procedure parameter extraction, importance value calculation and clustering to obtain an importance value heat map of the mutated model;
(S5) performing index calculation according to a test adequacy calculation method of the key elements of the black box testing and the key elements of the white box testing; and
(S6) evaluating indexes calculated in step (S5);
wherein the step (S2) comprises:
extracting a network parameter pi,j of each layer of a to-be-tested fully-connected neural network, wherein pi,j represents a parameter of a j-th layer under an i-th test case, and the network parameter pi,j comprises a weight matrix ωi,j and a bias vector bi,j, wherein dimensions of ωi,j and bi,j are determined by the number of neurons; setting the number of neurons in the j-th layer as m and the number of neurons in a (j−1)-th layer as n, such that ωi,j is a m×n matrix, bi,j is a m×1 matrix, and pi,j is a m×(n+1) matrix and expressed as pi,j=[ωi,j, bi,j]; and
extracting a test procedure parameter ci,j during the testing, wherein ci,j represents a test procedure parameter of the j-th layer under the i-th test case, and comprises maximum ui,j and minimum di,j of activating neurons in the j-th layer;
setting the number of neurons in the j-th layer as m, such that ui,j and di,j are both a m×1 matrix, and ci,j is a m×2 matrix, and expressed as ci,j=[ui,j, di,j].