| CPC H03M 7/3059 (2013.01) [G06F 21/554 (2013.01); G06N 20/00 (2019.01); H03M 7/6005 (2013.01)] | 10 Claims |

|
1. A system for data compression with intrusion detection, comprising:
a computing device comprising a processor and a memory;
an intrusion detection module comprising a first plurality of programming instructions stored in the memory which, when operating on the processor, causes the computing device to:
receive a codeword data stream;
use one or more algorithms to compute a probability distribution of a plurality of codewords within the codeword data stream;
compute an amount of divergence between the computed probability distribution and a reference probability distribution;
when the computed amount of divergence exceeds a configured risk sensitivity threshold, store the computed divergence, the computed probability distribution, and the codeword as anomalous event data in a database;
generate an intrusion alert, the intrusion alert comprising the anomalous event data; and
send the intrusion alert to a user interface to be viewed by a user;
a codebook training module comprising a second plurality of programming instructions stored in the memory which, when operating on the processor, causes the computing device to:
receive a training dataset;
use the training dataset to create the reference probability distribution;
send the reference probability distribution to the intrusion detection module;
receive data;
format the received data into a test dataset;
retrieve a first measured probability distribution associated with a previous training dataset from a monitor database;
use one or more algorithms to measure a second probability distribution of the test dataset;
compare the first and second measured probability distributions to compute the difference in distribution statistics between the test dataset and the previous training dataset;
determine when the difference in distributions exceeds a pre-determined difference threshold;
use the test dataset to retrain encoding and decoding algorithms;
utilize the retrained algorithms to create new data sourceblocks;
create new codeword for each new data sourceblock;
store each new data sourceblock and its associated new codeword in an updated codebook; and
send the updated codebook to a plurality of encoding and decoding machines.
|