Abstracts

Rationale: Recent advancements in implantable devices have led to various tools capable of EEG recording. These tools provide an unprecedented view into numerous neurological disorders. However, the volume and complexity of EEG recordings present significant challenges in efficient and accurate analysis. Previous methods attempt to automate the process of annotating features of interest but still require extensive manual annotation by experts to train a model initially, which is time-consuming and often impractical for large datasets. This paper proposes an automated seizure annotation and classification pipeline to reduce seizure annotation effort. The objective of this study is to introduce a ‘smart’ human-in-the-loop annotation pipeline that iteratively selects the most informative batch of data to annotate, improving classification performance and reducing the number of human annotations. Utilizing deep active learning and self-supervised learning, we developed a pipeline aimed at reducing manual effort and improving the efficiency of automated seizure annotation.


Methods: We used intracranial EEG recordings from 32 subjects from two sources of data (NeuroVista and Neuropace) to verify the performance and generalizability of the pipeline in different datasets. We used 1,500 hours of iEEG recordings without labels to train a ResNet-50 model using the self-supervised learning method SWaV to generate robust, dataset-specific feature embeddings. Various active learning strategies were implemented and tested to efficiently select and request human annotations from the unannotated data pool to train a multilayer perceptron for classification. We measured the model's performance using F1 score, and Cohen's Kappa score and benchmarked the machine learning model’s annotation accuracy against that of expert neurologists and training fellows. We also tracked the number of annotations used and their corresponding classification performance to select the best active learning strategy that achieves the best performance with the least number of annotations. We applied post-hoc explainability, thus providing a robust assessment of the model's predictive accuracy and confidence in its outputs.


Results: The proposed pipeline significantly improved annotation efficiency and classification performance, showing adaptability across multiple datasets. Its classification performance matches or exceeds that of human neurologists. The system achieved an F1 score of 0.947±0.012 on RNS data using around 320 short segments. The algorithm outperformed the first-place entry in UPenn and Mayo Clinic's Seizure Detection Challenge using only a fifth of the available data.


Conclusions: "Annotation Co-pilot" demonstrated expert-level performance, robustness, and generalizability across various datasets. This algorithm enables high-accuracy annotation at scale on new datasets. With further development, it will be an invaluable tool for streamlining the annotation of recordings, assisting neuroscience researchers and neurologists in research and diagnosis.


Funding: Ghost in the Machine: Melding Brain, Computer and Behavior, # 1-DP1-NS-122038-01
Guiding epilepsy surgery using network models and Stereo EEG, # 1-R01-NS-125137-01