Background: Traditional methods to elicit feedback on clinical decision support (CDS) tools lack specificity and actionable insights necessary to optimize hospitalist workflows and patient care (1,2). Alert fatigue—a significant issue among hospitalists—reduces provider satisfaction and jeopardizes patient safety by prompting clinicians to dismiss critical alerts (3).
Purpose: This initiative aimed to implement and evaluate a process for collecting real-time feedback on CDS alerts. The goal was to enhance the quality of the CDS system, improve clinician workflows, and ultimately advance patient care.
Description: In collaboration with the health system’s CDS and Clinical Informatics teams and the Hospital Medicine Clinical High Reliability Group, a custom five-question survey was created to collect real-time feedback on CDS alerts. The survey assessed the utility, usability, and clinical relevance of high-priority alerts selected based on firing frequency and workflow disruption potential.The initial phase, launched in late July 2024, embedded survey links into 7 specifically selected alerts to enable immediate feedback collection. After initiation of this phase, the EHR vendor announced a system-wide upgrade to integrate feedback links into all CDS alerts, approximately 400 in total. Consequently, the custom survey was discontinued.Between July 26, and October 2024, the custom survey collected 72 responses. After the vendor’s solution was implemented in October, 458 additional responses were gathered by November 18. To reassure clinicians that their feedback was being reviewed, the clinical informatics team sent a system-wide email in November, emphasizing that the CDS team was evaluating their input to guide alerting system improvements.
Conclusions: Embedding feedback links into alerts significantly increased the volume and quality of feedback compared to traditional methods like retrospective surveys or focus groups. The custom survey achieved the goal of capturing actionable, real-time insights from clinicians, providing quantitative data on alert usefulness, patient care impact, and clarity of recommended actions.While the vendor’s solution increased the volume of feedback, it relies on qualitative free-text responses. In contrast, the custom survey included quantitative questions designed based on the “Five Rights of CDS” framework to assess whether alerts delivered the right information, to the right person, in the right format, through the right channel, at the right time (4).In addition, the success of a system-wide alert feedback effort depends on evaluating and acting on feedback effectively. Failing to do so risks eroding trust in CDS systems and exacerbating alert fatigue. Developing robust processes to analyze and act on feedback is critical, though resource limitations in some hospital systems may pose challenges.Conversely, the vendor solution requires fewer resources for implementation and maintenance, offering a more cost-effective option. Hospitals should weigh these trade-offs when considering similar processes.Insights from this initiative also provide a framework for benchmarking CDS quality across systems. By analyzing trends in alert feedback such as usability and clinician perceptions, hospitals can identify areas for improvement and strengthen their CDS design. As predictive CDS tools driven by AI become essential, capturing meaningful feedback will remain crucial for reducing alert fatigue, optimizing workflows, and improving outcomes.
