US 12,413,563 B2
Method, device, and program product for inference
Tianxiang Chen, Shanghai (CN); Jinpeng Liu, Shanghai (CN); Wenqing Peng, Dezhou (CN); and Zhen Jia, Shanghai (CN)
Assigned to Dell Products L.P., Round Rock, TX (US)
Filed by Dell Products L.P., Round Rock, TX (US)
Filed on Feb. 7, 2024, as Appl. No. 18/434,964.
Claims priority of application No. 202311873051.0 (CN), filed on Dec. 29, 2023.
Prior Publication US 2025/0220006 A1, Jul. 3, 2025
Int. Cl. H04L 29/06 (2006.01); G06F 21/55 (2013.01); G06N 3/08 (2023.01); H04L 9/40 (2022.01)
CPC H04L 63/0435 (2013.01) [G06F 21/554 (2013.01); G06N 3/08 (2013.01); H04L 63/0442 (2013.01); H04L 63/045 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for inference, comprising:
determining, for a layer in a neural network model, whether a weight matrix of the layer is an irreversible target matrix;
partitioning, in response to determining that the weight matrix of the layer is the target matrix, the neural network model into a first part and a second part, wherein the first part comprises the layer and a previous layer before the layer and is deployed inside a security space, and the second part comprises a subsequent layer after the layer and is deployed outside the security space;
identifying, within the security space, at least one security space interior operator for at least one layer of the first part of the neural network model deployed inside the security space;
identifying, within a neural network framework, at least one security space exterior operator for at least one layer of the second part of the neural network model deployed outside the security space;
determining, within the neural network framework, one or more shadow operators corresponding to respective ones of one or more security space interior operators, wherein the one or more shadow operators and one or more security space exterior operators form a consecutive operator sequence in the neural network framework, and a last shadow operator in the one or more shadow operators is coupled to a first security space exterior operator in the one or more security space exterior operators; and
acquiring an inference result by using the partitioned neural network model based on encrypted inference data.