Rationale:
Persons with epilepsy are at a significantly increased risk of suicide compared to the general population. Effective suicide screening is the first line of defense against completed suicide. In the year prior to suicide, 80% of patients will see a healthcare provider at least once, and over 50% will be seen in the eight weeks prior to completing suicide. Despite the call for better suicide screening in PWE within the literature, there are no established suicide screening or prevention guidelines specific to the epilepsy population. While the NDDIE is superior to the PHQ-9 for detecting depression in PWE, it is well established that depression screening tools are inadequate for assessing suicide risk and lead to false negatives. In response to this unmet need, the Dartmouth epilepsy center set out to use machine learning to create a suicide screening tool for PWE within our clinic.
Methods:
A retrospective analysis of 3,184 patients seen at the Dartmouth Epilepsy Center between January 2013-January 2019 was performed. Data from intake tablets was analyzed including a full review of systems covering disease and psychosocial factors, including depression and SI, seizure frequency and severity, the Neurological Disorders Depression Inventory for Epilepsy (NDDIE), and quality of life rating. Vital statistics and cause of death data was pulled from the national death index. Two tree-based models, XG Boost and Light Gradient Boosting were used in the development of our predictive algorithm. The model was trained at the individual level to avoid data leak, and a 5-fold validation framework was employed.
Results:
In terms of predictive accuracy for completed suicide within our cohort; passive suicidal ideation (SI) defined as endorsing the statement “I would be better off dead”, was found to have an AUC of 0.52, active suicidal ideation “having thoughts of ending one’s life,” was found to have AUC of 0.62, NNDIE score was found to have an AUC of 0.53. Our predictive algorithm demonstrated, with increasing accuracy as encounters increased, an AUC of up to 0.82.
Conclusions:
Our ML algorithm which was able to retrospectively identify which patients will go on to complete suicide with an accuracy of up to 82% (AUC=0.82). By comparison, in the same cohort, the NDDIE showed only a 53% accuracy rate (AUC=0.53), performing only slightly better than chance, while reports of passive and active suicidal ideation has AUCs of 0.52 and 0.62, respectively. This represents a significant improvement in our ability to identify which patients are at high risk for impending suicide. If this degree of accuracy holds true when used prospectively, it could transform suicide prevention efforts in the epilepsy clinic.
Funding:
This project was supported by The Hitchcock Foundation at Dartmouth College