Background: Sepsis is a leading cause of death among hospitalized patients. Early detection of sepsis has the potential to reduce mortality by facilitating timely evidence-based interventions. Past studies have used electronic health records (EHR) to trigger alerts at the onset of sepsis, or to predict general clinical deterioration. In this study we describe the impact on clinical processes and outcomes of the first machine-learning algorithm created to predict severe sepsis or septic shock.
Methods: We developed and deployed a real-time machine-learning algorithm to predict patients at risk of developing severe sepsis and/or septic shock. We trained a random forest classifier using EHR data (such as labs, vitals, and demographics) from patients discharged between October 2011 and June 2014 (n=162,212 discharges). Positive cases (n=943) were defined as having ICD9 codes ‘995.92’ (Severe Sepsis) or ‘785.52’ (Septic Shock). Encounter data was sampled hourly and positive cases were labeled at 12 hours before onset of severe sepsis or septic shock (defined as a positive blood culture with either a lactate > 2.2 mmol/L or a systolic blood pressure < 90 mm Hg). We prospectively validated the algorithm by deploying it in “silent mode” from Oct to Dec 2015 (n=10,448 discharges, alerts=314). The positive likelihood ratio for severe sepsis/septic shock was 13. Starting June 2016, the system triggered an alert, which was sent to patients’ provider, nurse, and a rapid response coordinator. The team was then expected to perform an immediate bedside assessment for which no particular interventions were specified. We then compared outcomes between a pre-period (Jan – June 2016) and a post-period (June – Nov 2016) including frequency of clinical processes (e.g. lactate testing, intravenous fluid boluses, and antibiotics within 3 hours) and patient outcomes (e.g. ICU stay during hospitalization, severe sepsis/septic shock, mortality) for adult non-ICU inpatients.
Results: The tool triggered 1,816 times during the pre-period and 1,549 times during the post-period. The alert resulted in a modest but statistically significant increase in lactate testing (pre 8.0%, post 10.4%; p<0.01) and administration of IV fluid boluses (pre 23.3%, post 25.0%; p=0.02) within three hours of the alert. Initiation of antibiotics within three hours did not significantly differ between the pre- and post- periods. There was no statistically significant difference in the development of severe sepsis or septic shock, any ICU stay during hospitalization, or mortality.
Conclusions: An automated prediction tool for sepsis prompted diagnostic testing and early volume resuscitation, yet did not reduce severe sepsis/septic shock, ICU stays or mortality for those triggered. Next steps will include assessing clinician perceptions of the tool’s utility and timing, which may provide insight into the lack of effectiveness of the tool.