The efficient performance of an ED relies heavily on an effective triage system, which plays a crucial role in prioritizing patients based on the severity of their medical conditions and the urgency of treatment required [1]. The appropriate medical attention is critical, especially in life-threatening or time-sensitive emergencies where prompt intervention can significantly impact patient outcomes. One process crucial for the provision of these timely services in the ED is triage [2]. This is a process in which an initial clinical evaluation is conducted to select incoming patients who demonstrate an immediate demand for urgent care. The process typically uses a uniform scale to assess the severity of a condition before the physician assessment [3]. Many acuity-scoring systems have been developed to assess triage and the appropriate strategies for implementation in the ED environment [4]. One of the most commonly used models for triage is the CTAS, a widely adopted and standardized method used to categorize patients in EDs based on their clinical urgency [5]. The commonly used triage models is the CTAS, which is widely adopted internationally. However, in the United States, the Emergency Severity Index (ESI) is more frequently utilized. Both systems aim to categorize patients based on clinical urgency to optimize emergency department resource allocation [6].

However, traditional triage systems, including those using CTAS, may involve subjective assessments by healthcare providers, leading to potential inconsistencies and delays in patient care [7]. Different healthcare professionals may interpret patient’s symptoms differently, resulting in variations in triage decisions for patients with similar medical conditions. Such subjectivity can affect the accuracy of patient prioritization and resource allocation, potentially causing delays in critical cases or the unnecessary prioritization of less severe cases [8].

Traditional triage methods, such as the Emergency Severity Index and the Manchester Triage System, are prone to undertriaged and overtriaged, which can have a severe influence on patient outcomes and ED efficiency [9]. Undertriage, in which critically ill patients are incorrectly allocated lower acuity levels, can result in treatment delays and increased mortality risk. Overtriaged, on the other hand, causes lower-acuity patients to use key resources, which contributes to ED congestion [10]. Machine learning models are being developed to improve triage accuracy and speed patient flow.

In addition to the previous issues in traditional triage methods, we observed a notable lack of consulting retrospective data records of ED patients to reconsider decision-making [11]. One primary reason for this is the sheer size and complexity of available data. EDs often handle a large number of patients with diverse medical conditions, resulting in the accumulation of vast amounts of historical patient data over time [12]. We observed the lack of standardized methods and tools for analyzing this retrospective data which further complicated the decision making. The absence of robust data analytics platforms and expertise may also contribute to the underutilization of retrospective data in CTAS-based decision-making [13, 14].

Advancements in machine learning have led to the development of predictive models that often outperform traditional statistical methods in diagnosis and prognosis. Several ML models have demonstrated superior accuracy in predicting critical care outcomes, such as ED to intensive care unit (ICU) transfers and in-hospital mortality, compared to conventional screening tools like the Modified Early Warning Score, National Early Warning Score, and Sequential Organ Failure Assessment [15, 16]. In radiology, ML-based radiomics models have exceeded human performance in detecting subtle abnormalities that are often imperceptible to the naked eye. The practical implementation of the proposed model depends on its computational efficiency, seamless integration into clinical workflows, and clinician acceptance. However, models like Random Forest and SVM present interpretability challenges, which may hinder trust and adoption in emergency settings [17]. To enhance transparency, we utilized SHAP (Shapley Additive Explanations) to identify key clinical variables influencing triage predictions and LIME (Local Interpretable Model-Agnostic Explanations) to provide case-specific interpretations for decision support. Clinician acceptance can be strengthened through user testing of SHAP/LIME outputs, integration with electronic health record (EHR) systems for streamlined decision-making, and validation against expert physician assessments to ensure reliability [18, 19].

To address these challenges and improve the effectiveness of the triage process using retrospective data, this study proposes a Machine Learning (ML) approach for triage prediction at King Abdulaziz University Hospital (KAUH). The wealth of patient data available to hospitals via the ED’s systems is unmatched and can be used to create many applications that are useful in the ED context and can improve the management of the ED department and the allocation of hospital resources in a useful way.

Literature reported that the triage health care provider assessment in emergency care systems is difficult due to the growing number of patients and congestion. Traditional triage methods have issues with patient sorting and human error, which can risk patients’ lives. Machine learning (ML) technology can automate the triage decision-making process, resulting in more accurate and faster patient evaluations [20]. ML has demonstrated superior performance in predicting hospitalization and critical-care outcomes compared to reference triage models, possibly addressing overcrowding, enhancing health services, and lowering morbidity and death rates [21]. The literature reported the accuracy of a three-level triage system performed by triage nurses, and emergency medicine doctors in an ED. Data from 500 patients, including vital signs, primary complaints, age, and gender, were analyzed. Only 23.8% of patients received identical triage categorizations across all evaluators. Compared to emergency medicine doctors, triage nurses demonstrated slight overtriaged (6.4%) and undertriaged rates of 3.1% for yellow-coded and 3.4% for red-coded patients. Among AI models, demonstrated the highest accuracy but still undertriaged 26.5% of yellow-coded and 42.6% of red-coded patients. Given the significant undertriaged rates, AI models are not yet suitable for independent triage in emergency settings, requiring further optimization before clinical implementation [22].

The study conducted by Dugas et al. describes a computer-based electronic triage system (ETS) that optimises patient acuity distribution based on critical patient outcomes. The study evaluated the ETS to the Emergency Severity Index (ESI) in terms of patient distinction, outcomes, inpatient hospitalization, and resource utilization. The ETS dispersed patients more equally, identified patients with composite outcomes, and enhanced resource utilization. The ETS demonstrated a small improvement in patient distinction [23]. The study reported that e-triage more reliably detects ESI level 3 patients and emphasizes the potential of predictive analytics. The system predicts the requirement for critical care, emergent surgical intervention and inpatient hospitalization via a random forest model. At both EDs, e-triage outperformed the ESI in identifying clinical patient outcomes. E-triage detected more than 10% of ESI level 3 patients who needed up-triage and were at risk of critical care or an emergent surgical intervention [24].

To enhance patient triage in pediatric ED through the use of machine learning (ML) the study used a huge dataset of 189,718 patient visits over three years, with 9271 instances (4.98%) not hospitalized. Four machine learning models were tested: Deep Learning, Random Forest, Naive Bayes, and Support Vector Machines. The results demonstrated that ML prediction models trained on clinical outcomes performed better in triage than the present rule-based expert system. The study is among the first to use machine learning for pediatric ED triage [25]. A research in a Korean tertiary hospital attempted to predict early critical interventions (CrIs) for critically ill patients. The Extreme Gradient Boost (XGBoost) prediction model was utilized in the study, which had 137,883 patients. The model revealed that higher CrIs were related to worse ED outcomes. The CrIs model was incorporated into the site’s electronic medical record, allowing emergency physicians to propose early therapies [26]. Another study demonstrated that machine learning can reliably predict Korean triage Acuity Scale (KTAS) levels during triage reported to develop and compare machine learning models for predicting KTAS levels in ED. The random forest and XGBoost models exhibited the greatest AUROC, followed by clinical data-trained models [27].

Logistic regression is a prominent reference model for clinical triage prediction, however current research indicates XGBoost and deep neural networks as older techniques in terms of predictive accuracy. XGBoost is one of the best-performing triage classification models, whereas DNNs detect complicated non-linear patterns in clinical data. However, these models confront computational complexity and transparency issues, prompting more study into their incorporation into ED procedures [10].

The literature emphasizes the utility of machine learning in predicting clinical outcomes and dispositions in EDs. The reported research presented pediatric patients aged 18 years or younger who visited the ED Lasso regression, random forest, gradient-boosted decision tree, and deep neural network models were used. Their findings revealed that all machine learning algorithms had better discriminative ability for critical care and hospitalization, with fewer critically ill children undertriaged and fewer children overtriaged who did not require inpatient management. The decision curve study revealed that machine learning models provided a larger net benefit over a wide variety of clinical criteria [28]. The Manchester Triage System (MTS), a five-level triage system in Europe, categorizes patients based on symptom severity and urgency, aiming to prioritize timely care and optimize resource utilization, similar to CTAS and ESI. The effectiveness of ML-based prediction in executing the MTS was investigated utilizing data from Kepler University Hospital, in which RF and Neural Networks (ANN) were trained on the data to predict patient outcomes, such as discharge or admission for observation or intensive care. The results indicated that both RF and ANNs outperformed the other models in tasks, such as ward observation admission, intensive care admission, and 30-day mortality prediction [29].

ML-based triage prediction is dependent on the selection of relevant clinical parameters. Recursive feature removal and principal component analysis are two techniques that optimise feature sets during training. Cross-validation is a validation strategy that ensures model dependability. Resampling methods such as Synthetic Minority Over-sampling Technique (SMOTE) and ADASYN increase model performance at under-represented CTAS levels. Future study should investigate how these methods affect real-world triage accuracy and clinical decision support [10].

Previous research used machine learning to predict triage inside the CTAS framework. Hall et al. (2023) created an ML-based acuity score prediction model for virtual care environments [30], while Chen et al. (2023) used deep neural networks to predict important outcomes in ED patients [31]. However, these research were largely concerned with single prediction models rather than a comparative analysis of numerous ML algorithms. Furthermore, only a small amount of research has been conducted on using retrospective CTAS data analysis to improve triage accuracy in clinical settings. Our work fills this gap by creating and testing six ML models (KNN, SVM, DT, RF, GNB, and Light GBM) using a large retrospective dataset from King Abdulaziz University Hospital (KAUH), resulting in a complete health care provider assessment of ML performance in triage prediction.

Even systematic reviews reported that the implementation of ML models can contribute to better predictions of acuity scales. Literature focused on predicting patient’s need to access intensive care services. Among the ML models implemented in the reviewed studies are gradient boosting, logistic regression, neural networks, support vector machines, and random forests. The results indicated that Gradient Boosting, Logistic Regression, ANN and SVM demonstrated high performance in terms of accuracy ranges compared to other models [32].

Recent comprehensive evaluations have underlined the expanding importance of ML in emergency triage prediction. Sánchez-Salmerón et al. (2022) conducted a comprehensive study of ML approaches used in emergency triage, highlighting their potential to enhance decision-making and patient flow [3]. Similarly, Miles et al. (2020) examined ML-based risk prediction models and concluded that, while ML improves triage accuracy, model interpretability and incorporation into clinical practice remain problems [33]. More recently, Porto (2024) did a comprehensive study on the use of machine learning and natural language processing (NLP) in triage, revealing key research needs in data standardization and real-time deployment [10]. These studies underscore the need for more research to improve ML-based triage systems, particularly in pediatrics emergency situations.

Generally, these studies explore various ML models, and as the results indicate, there is no single model that outperforms every implementation of triage prediction, which means that the context of the study, type of data features, and size contribute to model performance. However, these studies helped in selecting the models that we wanted to explore in our experiment, including the SVM, RF, and KNN.

By leveraging ML techniques and algorithms, this study aimed to evaluate six ML models that objectively assessed patient’s acuity levels based on their clinical data. By minimizing subjectivity, these models seek to provide more accurate and consistent triage decisions, resulting in improved patient flow and optimized resource allocation within the ED. Therefore, the study trained six robust ML models for triage prediction in hospital EDs using the CTAS framework based on a large retrospective dataset from King Abdulaziz University Hospital and evaluate the overall accuracy of ML models for triage prediction on a dataset to determine the model with the highest accuracy and to evaluate the performance of ML models in the prediction of each CTAS level using the metrics of F1-Score, Precision, and Recall.