Background: Early warning scores are clinical decision support tools that incorporate multiple physiologic variables to detect patient deterioration. Previous studies have highlighted the value of integrating measures of provider intuition into predictive models. (1-3) For example, incorporating the Patient Acuity Rating, a Likert-based measure of nurse worry, into the six-variable Modified Early Warning Score (MEWS) improved its ability to detect cardiac arrest and emergent ICU transfer. (1) There has been rapid advancement in early warning scores in the last decade. It is unclear if intuition will continue to provide value when added to the electronic cardiac arrest triage score (eCART), a more sophisticated machine learning model used to predict clinical deterioration.
Methods: eCART combines 27 different lab values and vital signs to generate a patient-specific risk score between 0 and 100. Patients with scores 93-96 and ≥ 97 are considered at moderate and high risk of clinical deterioration, respectively. Clinicians caring for such patients are prompted to fill out a clinical pathway to indicate whether they believe the physiology is ‘stable/expected’ or they are ‘managing instability’. This was considered the nursing disposition variable and served as the measure of intuition in our study. We conducted a retrospective analysis examining all non-hospice adult patients on inpatient wards between January 2019 and August 2021 with available eCART and nursing disposition data. All dispositions and eCART scores between the first disposition and ward discharge were considered. MEWS scores were also calculated for each of these time points. AUCs were then calculated for models based on eCART score only, MEWS only, nursing disposition only, combined eCART/disposition risk score, and combined MEWS/disposition risk score. Combined risk scores were calculated by assigning a numerical value to the nursing disposition and equally weighting this value with the eCART or MEWS score. The outcome of interest was clinical deterioration (defined as cardiac arrest, ICU transfer, or death) within 24 hours of the nursing disposition or eCART score.
Results: The final sample included 14,265 patients. All available eCART scores, nursing dispositions, retroactively assigned MEWS scores, and combined risk scores were considered. AUCs were calculated on 4,430,190 observations and used to compare the predictive accuracy of five different models. eCART alone [0.78, 0.78-0.78] was more predictive of clinical deterioration than MEWS alone [0.72, 0.72-0.72]. The combined eCART/disposition model [0.75, 0.75-0.75] and MEWS/disposition model [0.71, 0.71-0.71] were less predictive than eCART or MEWS alone. A model solely based on nursing disposition [0.56, 0.56-0.56] was the least predictive. p-values for all comparisons were < 0.001.
Conclusions: Incorporating a binary measure of intuition into early warning scores, such as MEWS and eCART, did not improve their ability to detect clinical deterioration. Further research is needed to ascertain the value of incorporating more graded measures of intuition into early warning tools.