Abstracts

Rationale: About 50% of patients do not find their first anti-seizure medication (ASM) adequate. While randomized control trials are commonly used to evaluate the efficacy of medications, register data offers easily available data of national scale. In a previous study, we compared retention rate statistics from national registers to the expert ASM tool Epipick and concluded that if at least 50 patients were used per ASM in the registers, the two ASMs with the highest retention rate were suggested by Epipick in all test cases. This suggests that retention rate statistics from registers are a reasonable method to evaluate the treatment efficacy of ASM. However, there is reason to believe machine learning can achieve better performance. This project aims to use machine learning for personalized selection of ASM and find potential flaws in the current practice of selecting ASMs for patients.

Methods: Using Swedish national register data of ASM prescriptions of 38830 patients with epilepsy, we modeled how long patients used their first ASM and used it as a surrogate marker of severity of side effects and seizure suppression. A lower limit of 500 patients was allowed for each ASM, giving 7 ASM options. Thirteen different comorbidities were derived from ICD-codes. Two machine learning models (Counterfactual survival analysis [CSA], and Survival Causal inference [SurvCI]) that predicts the counterfactual survival rate (in this case of an ASM) were modified to fit our setting with multiple treatments. Since the real-world observational data contain selection bias, which would not be present if the model was used at a clinic, we created a synthesized data set to evaluate the performance from simulated register data and tested them in a setting of a randomized control trial. The method with the highest performance was then used to generate suggestions comparing to the current treatment policy, estimated as the most common treatment for patients of a specific sex, comorbidities, and age +- 2 years.

Results: We compared the two suggested machine learning models, two baseline machine learning models, and retention rate statistics and measured performance in concordance index (CI). Synthesized data set; randomized control trial CI (observational test data performance CI): Multi-CSA 0.705 (0.695), Multi-SurvCI 0.695 (0.705), Gradient Boost: 0.687 (0.700), Survival forest: 0.566 (0.593), Retention rate statistics: 0.633 (0.587). The Multi-CSA and Multi-SurvCI model were then applied to estimate the optimal treatment policy and compared to the current policy. The suggestions of the best treatment were often clearly not the best one for the chosen sex, age, and comorbidities.

Conclusions: While Multi-CSA and Multi-SurvCI outperform retention rate statistics, a model with similarity in performance to Epipick, they do not seem to be able to select the best treatment even though it should perform better in general. The reason for lack in performance could be, for example, that the machine learning model is good in general at ranking treatments for a specific patient, but bad at finding the best one.

Funding: Wallenberg Center for Molecular and Translational Medicine