| CPC A61B 5/1128 (2013.01) [A61B 5/4094 (2013.01); A61B 5/7264 (2013.01); A61B 5/7405 (2013.01); A61B 5/742 (2013.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 20/46 (2022.01); G06V 40/20 (2022.01); G06V 20/44 (2022.01)] | 10 Claims |
|
1. A method for detecting a motor seizure type of an epileptic seizure of a patient, said method to be performed by a patient monitoring system comprising a processor operatively connected with
a video camera,
a depth sensor or a stereoscopic imaging equipment,
an audio sensor, and
one or more databases comprising a first pre-trained neural network, a second pre-trained neural network and a third pre-trained neural network, the method comprising:
receiving, by the video camera, video data of the patient;
detecting, by an apparatus operatively connected to the video camera, a first anomaly in the video data as anomaly in movement of the patient, wherein the first anomaly in the video data is detected based on the video data, and the detecting comprises:
determining, by the apparatus, an actual feature over a video data segment, the actual feature representing actual movement of the patient;
determining, by the apparatus, a predicted feature over the video data segment using the first pre-trained neural network, wherein the first pre-trained neural network is pre-trained for normal sleeping data captured from persons sleeping without any seizures, the predicted feature representing predicted movement of the patient;
determining, by the apparatus, a difference between the actual feature and the predicted feature; and
registering, by the apparatus, the actual feature as the first anomaly based on the difference; and the method comprises:
determining, by the apparatus, a video frame stack comprising the first anomaly;
classifying, by the apparatus, the video frame stack using the first pre-trained neural network, that is a pre-trained image neural network, to obtain a first classification, wherein the first classification is based on a seizure classification for epileptic seizures;
receiving audio data from the audio sensor configured to detect sounds produced by the patient over time;
transforming the audio data to obtain a two-dimensional representation of the audio data;
determining a second anomaly in the two-dimensional representation of the audio data as anomaly in sounds produced by the patient based on the audio data or based on the detected first anomaly;
determining an audio clip comprising the second anomaly;
classifying the audio clip using the second pre-trained neural network, that is a pre-trained audio neural network, to obtain a second classification, wherein the second classification is based on the seizure classification for epileptic seizures;
receiving depth data from the stereoscopic imaging equipment or the depth sensor configured to detect movement of the patient over time;
detecting a third anomaly in the depth data as anomaly in movement of the patient;
determining a depth data frame stack comprising the third anomaly;
classifying the depth data frame stack using the third pre-trained neural network, that is a pre-trained depth neural network, to obtain a third classification, wherein the third classification is based on the seizure classification for epileptic seizures; and
determining, by the apparatus, the motor seizure type based on the first classification, the second classification, and the third classification by:
applying pooling to respective outputs of the pre-trained image neural network, the pre-trained audio neural network, and the pre-trained depth neural network to obtain a combined output;
determining the motor seizure type based on the combined output; and
generating, for display on a graphical user interface of a user device operatively connected to the processor, an interactive report comprising the motor seizure type determined based on the combined output, wherein the interactive report further comprises a plurality of motor seizure types classified by the patient monitoring system.
|