Authors :
Presenting Author: Zhibin Chen, PhD, CStat – Monash University
Shani Nguyen, Medical Student – Medical Student, Department of Neuroscience, Central Clinical School, Monash University; Wei Feng, MS – PhD Student, 2. Department of Electrical and Computer Systems Engineering, Monash University; Richard Shek-kwan Chang, MD – PhD Student, Department of Neuroscience, Central Clinical School, Monash University; Daniel Thom, BSc(Hons) – Study Coordinator, Department of Neuroscience, Central Clinical School, Monash University; Haris Hakeem, MD – PhD Student, Department of Neuroscience, Central Clinical School, Monash University; Zongyuan Ge, PhD – A/Professor of Medical AI, 2. Department of Electrical and Computer Systems Engineering, Monash University; Patrick Kwan, MD, PhD – Professor of Neurology, Department of Neuroscience, Central Clinical School, Monash University
Rationale:
We previously developed a deep learning model using a transformer architecture to predict treatment outcomes of the first antiseizure medication (ASM) for individual adults with newly diagnosed epilepsy.
1 The model was trained and internally validated in a pooled cohort of 1798 patients from Glasgow, UK; Kuala Lumpur, Malaysia; Chongqing and Guangzhou, China; and Perth, Australia. It demonstrated the feasibility of personalized prediction of response to ASMs based on clinical information. Whether the model can achieve similar performance in other populations is yet to be tested.
1 Hakeem H, et al. JAMA Neurol 2022;79(10):986-996.
Methods:
We reviewed the medical records of patients who attended the First Seizure Clinic at a tertiary hospital in Melbourne, Australia, between 2018 and 2020. Patients who fulfilled the same eligibility criteria as the development study were included as the validation cohort. The same sixteen clinical features and first ASM information that were used to train the model were extracted. The model predicted each individual patient’s probability of achieving seizure freedom within the first 12 months of commencing their first ASM regimen. As per cut-off used in the development study, a predicted probability >0.5 was classified as successful, i.e., seizure-free within the first 12 months. The predicted outcome was compared with the observed outcome to evaluate the performance of the model based on area under the operating characteristic curve (AUC), sensitivity, specificity, and weighted balance accuracy. 95% confidence intervals (CI) were determined via five fold cross-validation. Subgroup analysis in different epilepsy types, i.e., focal and generalized/unclassified, was performed.
Results:
A total of 91 newly diagnosed and treated adults with epilepsy were included in the validation cohort (65% male, median age 39 years) (Figure 1). Compared to the pooled development cohort, the validation cohort had somewhat different demographic and clinical characteristics (Table 1). Near half (48%) of the validation cohort commenced levetiracetam as the first treatment, compared to 17% in the pooled development cohort. In predicting the first ASM treatment outcome at one year in the validation cohort, the model has an AUC of 0.63 (95% CI: 0.61-0.65), sensitivity 0.62 (95% CI: 0.61-0.63), specificity 0.54 (95% CI: 0.52-0.56), and a weighted balance accuracy 0.58 (95% CI: 0.57-0.59). The model showed a better performance in predicting treatment outcomes in patients with focal epilepsy (AUC=0.67, 95% CI: 0.64-0.70), compared to those with generalized or unclassified epilepsy (AUC=0.59, 95% CI: 0.58-0.60).
Conclusions:
The deep learning model demonstrated modest but consistent performance in predicting treatment outcomes of the first ASM in an external validation cohort that had different demographic and clinical characteristics to the pooled development cohort. It shows the generalizability of the model using clinical features. However, further improvement in performance is necessary to enhance the model's clinical utility, possibly by incorporating genetic data and more detailed EEG and neuroimaging diagnostic information.
Funding: No