Background:

Failure and delay in identifying clinical deterioration in hospitalized ward patients can result in adverse outcomes. Although many hospitals have instituted rapid response teams to quickly treat and triage unstable patients, early and reliable identification of high‐risk patients remains a challenge. Objective, vital sign–based risk prediction scores, such as the Modified Early Warning Score (MEWS), have been developed for this purpose but can result in high false‐positives because of limited specificity. The Patient Acuity Rating (PAR), a subjective 7‐point Likert‐scale assessment of the likelihood of cardiac arrest (CA) or emergent transfer to an intensive care unit (ICU) within the next 24 hours has demonstrated good preliminary results. However, it has not been externally validated or directly compared with an objective metric such as the MEWS.

Methods:

From September 2011 to August 2012, hospitalists at an urban university hospital prospectively assigned a blinded PAR score to patients at the end of the day shift as part of the completion of a standard electronic medical record handoff (Cerner, Kansas City, MO). Completion of the PAR score was voluntary. Vital signs were abstracted for each patient from the hospital's enterprise data warehouse and a MEWS calculated using the vital sign set closest to the time of PAR assessment. Data on adverse events in the next 24 hours (defined as CA or emergent ICU transfer) were collected from the quality assurance database. Areas under the receiver operator characteristics curves (AUROC) were calculated for both the PAR and the MEWS to predict adverse events and compared using a paired analysis. A secondary analysis assessed the impact of the PAR score on the predictive accuracy of the MEWS.

Results:

Twenty‐eight hospitalists consented to participate and provided a total of 7419 PAR scores on 3344 distinct patients, with a median of 2. A corresponding MEWS was available for 7332 assessments, with a median of 3. There were 65 ICU transfers and 2 CAs during the study period. There was no significant difference between the PAR and MEWS AUROCs (0.64 [95% CI, 0.57–0.75] vs. 0.69 [95% CI, 0.63–0.75], P = 0.16). The MEWS AUROC at PAR thresholds of ≤3, 4 or 5, and ≥6 were 0.61 [0.52–0.69], 0.72 [0.62–0.83], and 0.89 [0.80–0.98], respectively (see Figure).

Conclusions:

Our validation demonstrated a PAR AUROC that was similar to the MEWS but lower than the original unblinded validation study (in which general medicine attendings demonstrated a PAR AUROC of 0.84). The predictive accuracy of the MEWS increased with increasing PAR, suggesting that the combination of the 2 could improve sensitivity and specificity for detecting adverse events and allow for better allocation of resources to high risk patients with fewer false alarms.

Figure.AUROCs of PAR, MEWS, and MEWS at PAR thresholds. Abbreviations: AUROC, area under receiving operator characteristic curve; CI, confidence interval; MEWS, modified early warning score; PAR, patient acuity rating