Authors :
Presenting Author: Zoë Tosi, PhD – Epitel, Inc.
Mitchell Frankel, PhD – Epitel, Inc.
Tyler Newton, PhD – Epitel, Inc.
Avi Kazen, MS – Epitel, Inc.
Vamshi Muvvala, MS – Epitel, Inc.
Rationale:
Extended-duration seizure monitoring is increasingly accessible with the advent of easily wearable EEG devices, necessitating automated seizure detection algorithms that reduce the review burden on epileptologists. However, to be clinically valuable, algorithms must identify electrographic seizures with a proficiency approaching that of experts. Seizure detection algorithms are often not evaluated in a way that directly addresses this standard; during validation, “ground truth” is typically established via consensus, limiting the interpretability of results, introducing bias, and sidelining the well-established phenomena of inter-expert divergence. Alternative Inter-rater (IR) evaluation schemes potentially address these shortcomings and the question of expert equivalence. Inter-expert agreement is explicitly quantified in IR, with algorithm performance reported relative to that agreement. Using IR analysis, the performance of the FDA-cleared REMI VigilenzTM AI for Event Detection (VED; designed for use with the reduced-channel REMITM Remote EEG Monitoring System) is reported relative to expert epileptologists. We further contextualize these findings by including a widely used 19+ channel algorithm in the analysis to aid in generalizing our results beyond our dataset.Methods:
Sixty EEG recordings (mean duration: 67 hours) from epilepsy monitoring units and at-home ambulatory settings were independently annotated for electrographic seizures by groups of three expert epileptologists and two algorithms. Experts and a state-of-the-art seizure detection algorithm reviewed the complete 19+ channels of 10-20 EEG, while VED operated exclusively on just four differential EEG channels (equivalent to bilateral frontal and temporoparietal placement). Positive Percent agreement (PPA) and false positives per day (FPs/day) were computed across expert-expert and expert-algorithm pairings.
Results:
Human experts produced a total of 348 markings across the 4,036 hours of data. VED averaged a PPA of 76.8% (95% CI=69.7%-83.8%) with 4.79 FPs/day (95% CI=3.69-6.15) relative to human experts. Average PPA between experts ranged from 69.4% (95% confidence interval [CI]=50.1%-85.6%) to 88.3% (95% CI=79.4%-87.4%), while average FPs/day ranged from 0.15 (95% CI=0.05-0.29) to 0.80 (95% CI=0.09-1.85). The state-of-the-art 19+ channel algorithm achieved an average PPA of 65.3% (95% CI=55.8%-74.4%) with 1.29 (95% CI=0.97-1.68) FPs/day. Conclusions:
Our findings show that REMITM Vigilenz AI for Event Detection, operating on only four channels, approaches human expert performance for detecting electrographic seizures and compares well to a 19+ channel state-of-the-art algorithm. While traditional consensus-based performance-goal approaches may be useful when comparing two algorithms, the IR methods applied here speak directly to the notion of expert equivalence, which may be more important for clinician trust. These results suggest that the VED algorithm may offer reliable automated review of reduced-channel EEG, enhancing epilepsy monitoring and care.Funding:
NIH 1U44 NS121562