Abstracts

Rationale: A detailed analysis of seizure semiology is critical for the management of epilepsy patients. Inter-rater reliability using qualitative visual analysis is often poor for semiological features. Therefore, automatic and quantitative analysis of video-recorded seizures is needed for objective assessment. An important characteristic is whether a seizure generalizes, as this increases the risk of injury and sudden unexpected death in epilepsy (SUDEP) significantly.

Methods: We present GESTURES (Generalized Epileptic Seizure classification from video-Telemetry Using REcurrent convolutional neural networkS), a novel deep learning architecture. GESTURES combines convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to learn representations of arbitrarily long videos of epileptic seizures. Seizure videos are divided into n segments of equal duration (Fig. 1a). A short snippet (≈ 0.5 s) is randomly sampled from each segment using a Beta probability distribution (Fig. 1b). We use a spatiotemporal CNN (STCNN), pre-trained on human action recognition (HAR) datasets comprising over 65 million video clips from Instagram and YouTube, to extract features from each snippet. We then train an RNN to learn seizure-level representations from the sequence of extracted features. We curated a dataset of seizure videos from 68 patients who underwent video-EEG at the National Hospital for Neurology and Neurosurgery between 2015 and 2019 and evaluated GESTURES on its ability to classify seizures into focal unaware seizures (FOSs) (N = 106) vs. focal to bilateral tonic-clonic seizures (TCSs) (N = 77). All patients gave written informed consent to participate in research to evaluate SUDEP risk (ethics approval 19/SW/00071).

Results: The highest accuracies were obtained using 16 segments, and bidirectional long short-term memory (BLSTM) units for aggregation (Fig. 2). The model with the highest accuracy (98.9%) and F1-score (98.7%) yielded 77 true positives, 104 true negatives, 2 false positives and 0 false negatives, where TCS is the positive class.

Conclusions: We demonstrate that an STCNN trained on a HAR dataset can be used in combination with an RNN to accurately represent arbitrarily long videos of seizures. GESTURES can provide accurate seizure classification by modeling sequences of semiologies. GESTURES is robust to the presence of multiple people in the room and to occlusions by nurses or sheets, and able to analyze videos with different fields of view and camera types (RGB, infrared). Such automated classification could support video analysis in clinical settings and provide objective tools to classify seizures in research settings.

Funding: Please list any funding that was received in support of this abstract.: Academy of Medical Sciences Springboard (BF005\1131). NPIF EPSRC Doctoral (EP/R512400/1). National Institutes of Health - National Institute of Neurological Disorders and Stroke (Center for SUDEP Research; U01- NS090407). Computing infrastructure at the Wellcome / EPSRC Centre for Interventional and Surgical Sciences (WEISS) (UCL) (203145Z/16/Z) was used for this study.