Enhancement of In-hospital Mortality Prediction in EmergencyDepartment using Ensemble Machine Learning Models

Early prediction of severity level in critically ill patients and their outcome, definitely can be affected on prioritizing patients and declining in-hospital mortality in emergency department. Several classical Scoring systems have been proposed for indication of severity of patient illness, such as logistic regression models. However, as an alternative approach, one of the main streams in machine learning methods, known as ensemble models, are proposed in the current study to compare their modern models with the old ones. Hence, the aim of this study is evaluation and comparison of the traditional model’s logistic regression (LR) and modern ones (Bagging, AdaBoost, Random Forest (RF), and Extreme Gradient Boosting (XGB), and detection of the model with the best predictive performance for prediction of in-hospital mortality Methods:An observational single-center study was conducted in the Emergency Department (ED) of Imam Reza Hospital from March ۲۰۱۶ to March ۲۰۱۷ which is located in the northeast of Iran. Adult patients with one to three level of ESI acuity was defined as meeting criteria for this study. The training and validation visits from the ED were randomly divided into ۸۰% vs ۲۰%. After training the models using ۱۰-fold cross-validation on the training set, their predictive performance was then evaluated.Results:the cohort consisted of ۲,۰۲۵ unique patients admitted to the ED of the hospital. There were about ۱۹% of hospital deaths. Of the ۱,۴۷۶ patients in the training group, ۲۷۴ (۱۸.۶%) died during hospitalization, and of the ۷۲۸ patients in the validation group, ۱۵۲ (۲۰.۸%) died during hospitalization. The AUCs with ۹۵% confidence intervals (CIs) for the di↵erent models were as follows: the RF, ۰.۸۱۲ (۹۵% CI = ۰.۷۴۲ to ۰.۸۳۰); XGB, ۰.۷۹۸ (۹۵% CI = ۰.۷۵۶ to ۰.۸۴۰); Bagging, ۰.۷۹۶ (۹۵% CI = ۰.۷۵۳ to ۰.۸۳۷); Adaboost, ۰.۷۹۱(۹۵% CI = ۰.۷۴۵ to ۰.۸۳۷); and the LR, ۰.۷۸۶(۹۵% CI = ۰.۷۴۲ to ۰.۸۳۷). Among all ML-based prediction models the AUC of the RF was statistically di↵erent from the LR (p ≤ ۰.۰۲۷۵). The AUPRC was ۰.۵۷۷(۰.۵۴۱-۰.۶۱۲) for the RF, ۰.۵۵۷(۰.۵۲۰-۰.۵۹۳) for the XGB, ۰.۶۰۵(۰.۵۶۹۰.۶۴۰) for the Bagging, ۰.۶۰۳(۰.۵۶۷-۰.۶۳۸) for the Adaboost, and ۰.۵۹۲(۰.۵۵۶-۰.۶۲۷) for the LR. All models had inadequate calibration. The most accurate models belonged to the RF and XGB with lowest BS. Conclusions: the RF can be used as a screening tool to identify patients at risk of mortality. This tool could improve the management of hospital resources and patient-throughput planning, thus delivering more effective care to hospitalized patients.

کلیدواژه ها:

Ensemble machine learning ، In-hospital mortality ، Emergency department ، Logistic regression ، Bagging ، AdaBoost ، Random Forest ، and Extreme Gradient Boosting ، Unbalanced data ، Feature selection