Background: An estimated 20-30% of inpatient standing labs are medically unnecessary1, contributing to iatrogenic anemia2, sleep disruption3, and increased healthcare costs. The Society of Hospital Medicine identified routine repetitive complete blood count (CBC) and chemistry (BMP) as a common wasteful inpatient practice4. At our institution, repetitive inpatient CBC, BMP, magnesium, and phosphorus account for 500,000 tests annually. Clinicians cite cognitive burden and safety concerns as barriers to discontinuing daily labs.
Methods: We developed a probabilistic regression model using 7 years of inpatient data from our institution to predict the probability of the next lab falling in a predefined stability range5. We interviewed 20 clinicians from various inpatient specialties to design the user-facing alert (Fig. 1) and define consensus stability ranges for CBC and BMP components. “SmartAlert” (Fig. 2) sends real-time EHR data to the prediction algorithm every 6 hours, stores the result in the EHR, and references this value to determine if the alert should be displayed to providers ordering labs with daily or more frequency. On July 25, 2024, the CBC-focused SmartAlert was deployed at our institution to all inpatients, excluding Bone Marrow Transplant and ICU patients. The alert was randomized on the patient level to either display when trigger criteria were met or to silently log the alert without displaying. Alert logs and EHR data were used to evaluate the prediction algorithm’s performance and the impact of the displayed alert on physician ordering behavior.
Results: As of November 5, 2024, the algorithm has screened 9537 unique patients and generated 311,012 predictions. Of these, 74% carried a stable prediction. When compared to the next available CBC result within 28 hours of the prediction, the algorithm’s sensitivity was 77%, specificity 44%, negative predictive value 28%, and positive predictive value 87%. Ensuring a high PPV (how often was a lab stable after a stable prediction) was particularly important as missing a potentially unstable lab poses the most harm to the patient. The interruptive alert displayed 213 times to 96 unique clinician users involving 146 unique patients, whereas the silent alert triggered 998 times on 489 unique patients. 66% patients in the silent arm had a CBC result within 28 hours of the alert compared to 47% in the displayed arm, a 29% relative reduction in repeat testing. There was no free-text user feedback provided on the alert. The most common reason for dismissing the alert was medical necessity. Qualitative analysis of free-text dismissal reasons revealed peri-procedural testing, medications impacting blood counts, monitoring of clinical, and provider deemed unstable labs as other situations for dismissing the alert.
Conclusions: A SmartAlert that leverages machine learning to encourage discontinuation of potentially medically unnecessary labs was effective in reducing subsequent CBC testing in this randomized controlled deployment.
