Background: Best-Practice Alerts (BPAs), utilized to reduce the number of adverse events in medical settings, have limited efficacy partly due to alert fatigue and user desensitization. Alert fatigue often stems from inaccurate alerts firing too often as well as poor physician perception of the efficacy of the alerts, both of which lead to reduced physician response. This study aimed to develop a sepsis alert with improved accuracy, effectiveness, and clinician satisfaction to reduce alert fatigue.
Methods: To assess baseline accuracy of the previous sepsis alert, a chart review was conducted on 200 randomly selected charts on which the prior sepsis alert had fired. The original sepsis alert fired if two Systemic Inflammatory Response Syndrome (SIRS) criteria were present in a patient greater than 13 years old. The updated alert was designed to mirror actual clinical sepsis criteria, including two SIRS criteria, one sign of end organ damage, and the pre-requisite that the sepsis order set had not already been requested. Baseline clinician alert fatigue and satisfaction was assessed via paper survey of physicians, physician assistants, and residents one month prior to the alert update. The results of this baseline survey were compared to a repeated survey administered three months post-update. Alert fatigue was evaluated by comparing the number of times physicians who encountered the alert in the 90 days before and after the update. Effectiveness of the alert was determined by comparing the percentage of patients for which clinicians placed the sepsis order set after encountering the alert in the three months before, and the three months after the alert update.
Results: In the 90 day period prior to the alert update, the alert fired 4085 times on 1000 patients. During this time, 3592/4085 alerts were cancelled/timed out (87.9%), 286/4085 were overridden (7.0%), and sepsis orders were placed on 207/1000 patients (20.7%). Chart review revealed that 30/200 patients on which the original sepsis alert fired actually met clinical sepsis criteria for a baseline accuracy of 15%. Following the update, the alert fired 172 times on 36 patients, 95.8% less than the previous alert. Of these, 154/172 were cancelled/timed out (89.5%), 5/172 were overridden (2.9%), and sepsis orders were placed on 13/36 patients (36.1%). The percentage of patients on which orders were placed under the new alert criteria was found to be significantly higher post-update (p=0.0267). Comparison of the pre- and post-update provider surveys revealed no significant change in opinion related to how often the alert was encountered, likelihood of acceptance, confidence in the alert, understanding of the criteria, contribution to patient care, or overall satisfaction (Figure 1).
Conclusions: The previous sepsis BPA was found to have very poor accuracy in identifying patients with clinical sepsis, a problem solved by basing the new alert on clinically-acceptable sepsis criteria. While providers were more likely to order testing for a patient flagged by the BPA, an action rate of 36.1% is still very poor. This may be partially explained by the continued presence of other, less accurate BPAs maintaining a high alert burden for providers. While the incidence of alert firing decreased and the accuracy improved, providers still did not perceive the new alert as being more efficacious or beneficial to patient care. Much of this gap in perception could be addressed through face-to-face clinician education and more prominent notification of alert criteria updates.
