Background: Clinically stable inpatients may receive potentially unnecessary care, such as overnight vital sign assessment. Nighttime vital signs can disrupt sleep and adversely affect patient satisfaction and contribute to delirium. However, it may be difficult for individual clinicians to determine which patients could safely forego overnight vital signs.
Purpose: We developed a predictive algorithm designed to identify patients who would likely have stable nighttime vital signs and created a clinical decision support tool embedded in the electronic health record (EHR) to prompt clinicians to reduce vital sign checks for those patients.
Description: Predictive Model Development and Validation
We extracted six months’ worth of data from our Epic-based EHR on 3519 inpatients, encompassing a total of 20,835 encounter-nights. Data elements included vital signs, laboratory tests, and clinical data, such as patient demographics and orders written. The primary outcome of the model was “abnormal nighttime vital sign”, defined as any nighttime vital sign entry (blood pressure, heart rate, respiratory rate, or O2 saturation) not within the standard call parameters. We trained a multivariable logistic regression model to predict whether that encounter-night would have ≥1 abnormal vital sign based on the clinical and patient-level variables from the day prior to the encounter-night. Training was performed on a random 70% of the data set (“training set”). We tuned the model on the other 30% (“validation set”) and aimed for a high positive predictive value (>90%) to minimize the chance of missing abnormal vital signs and potentially causing harm. The final independent variables contained in the model were: age, hospital day, prior night’s vital signs, current day’s vital signs, and the current day’s white blood cell count, hemoglobin, creatinine, bicarbonate, and potassium lab values.
The data processing and model development were performed using the R programming language. In order to implement our predictive model into the Epic EHR, we converted it into the Predictive Model Markup Language (PMML) format and then uploaded it into Epic’s Cognitive Computing module, a newly available feature in Epic (Figure 1).
Alert Development
We designed a novel side-bar alert that would 1) notify a clinician if their patient was identified by our model as having a >90% chance of having stable nighttime vital signs, 2) provide the scoring that led to this prediction, 3) suggest reducing vital sign checks if the clinician agreed, and 4) allow the clinician to reduce vital sign checks from within the alert itself. The alert was purposely unobtrusive and allowed clinicians to continue their usual workflow without being forced to respond to it immediately (Figure 1).
Pilot-testing and Implementation
The side-bar alert was assessed for feasibility and acceptability by pilot-testing it for 4 weeks on the direct-care hospitalist service of our 600-bed academic teaching hospital. We surveyed 10 attending hospitalists on service during the pilot, and of the 8 that reported receiving the alert, 75% (n=6) found it useful to their practice.
Conclusions: We developed a reliable predictive model that could identify patients with stable nighttime vital signs and created a clinical alert to encourage clinicians to order less frequent vital sign checks on these patients. Initial pilot-testing indicated reasonable acceptability among hospitalists. Future directions include assessing the effectiveness and safety of the alert on hospitalized patients.
