US 11,783,201 B2
Neural flow attestation
Zhongshu Gu, Ridgewood, NJ (US); Xiaokui Shu, Ossining, NY (US); Hani Jamjoom, Cos Cob, CT (US); and Tengfei Ma, White Plains, NY (US)
Assigned to International Business Machines Corporation, Armonk, NY (US)
Filed by International Business Machines Corporation, Armonk, NY (US)
Filed on Jan. 23, 2020, as Appl. No. 16/750,328.
Prior Publication US 2021/0232933 A1, Jul. 29, 2021
Int. Cl. G06N 3/10 (2006.01); G06N 3/08 (2023.01); G06N 3/04 (2023.01)
CPC G06N 3/10 (2013.01) [G06N 3/04 (2013.01); G06N 3/08 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method, in a data processing system comprising at least one processor and at least one memory, the at least one memory comprising instructions that are executed by the at least one processor to configure the at least one processor to implement a neural flow attestation engine, the method comprising:
inputting, by the neural flow attestation engine, input data to a trained computer model, wherein the trained computer model comprises a plurality of layers of neurons;
recording, by the neural flow attestation engine, for a set of input data instances in the input data, an output class generated by the trained computer model and a neural flow through the plurality of layers of neurons to thereby generate recorded neural flows, wherein the output class is one of a plurality of possible output classes;
deploying the trained computer model to a computing platform; and
verifying, by the neural flow attestation engine, an integrity of the deployed trained computer model based on a runtime neural flow of the deployed trained computer model and the recorded neural flows, wherein the verifying comprises:
determining deviations between the runtime neural flow and one or more recorded neural flows corresponding to a same output class as generated by the deployed trained computing model for the runtime input data; and
determining that the execution integrity of the deployed trained computer model has been compromised in response to the deviations satisfying a predetermined criterion.