DELTA: Changing the Way to Accurately and Promptly Identify Patient Deterioration

Background: Failure to promptly recognize and intervene upon patient deterioration leads to morbidity and mortality. Numerous “early warning scores” (EWS) intend to identify such patients but their performances lack adequate sensitivity and specificity. “DELTA” was designed to automate the methodical thought-process and trend analysis of a human provider and to distinguish itself by leveraging relative rather than the static analysis commonly used in EWSs. DELTA analyzes a combination of 46 data points (10 vitals and 36 commonly obtained labs) across three domains – proportion of “good” (a data point outside normal limits and trending toward normal limits) to “bad” (a data point outside normal limits and trending away from normal limits) change; relative magnitude of change; and change relative to the patient’s baseline. When three domains suggest a deteriorating patient, DELTA fires an alert.

Methods: DELTA’s performance was analyzed using 24 historic Mayo Clinic Rochester patient hospitalizations. An “event” was defined as an RRT/ICU transfer, CODE 45/CPR, death, positive bacterial or fungal culture, imaging revealing acute pathology, and/or surgery. Charts were reviewed to determine event times. DELTA was prospectively simulated on the datasets. A positive alert was defined as occurring 0-72 hours prior to an event, except for surgical events for which positive alerts were defined as occurring 0-72 hours after surgery. DELTA’s performance was evaluated through sensitivity (SEN), specificity (SPEC), positive and negative predictive value (PPV, NPV), and positive and negative likelihood ratio (LR+, LR-) analysis.

Results: DELTA’s overall performance demonstrated SEN=64.5%, SPEC=98.7%, PPV=70.2%, NPV=98.3%, LR+=49.4, LR-=0.360. When restricting events to RRT/ICU transfer, CODE 45/CPR, and death, metrics showed SEN=72.4%, SPEC=98.2%, PPV=47.7%, NPV=99.4%, LR+=41.0, LR-=0.281. When events occurring within 72 hours of admission were discounted (to grossly account for dataset limitations), overall metrics showed SEN=88.9%, SPEC=98.7%, PPV=70.2%, NPV=99.6%, LR+=68.1, LR=0.113. Of missed events, 77% and 41% occurred within 72 and 24 hours of admission, respectively. DELTA on average alerted 14, 39, 50, 36, and 30 hours prior to patient RRT/ICU transfer, CODE 45/CPR, death, bacteremia report time, and positive imaging, respectively. There were 62 events. Fifteen patients had at least one event.

Conclusions: DELTA demonstrates promise in improving patient deterioration recognition through a novel algorithm. DELTA compares favorably to the commonly used “MEWS” and Rothman Index (RI) in alerting within 24 hours of death with SEN=63.6%, SPEC=97.5% relative to SEN=49.8%/48.9%, SPEC=93.6%/97.1% for MEWS=4 and RI=16 thresholds, respectively. Relative to a high-performing neural-network model, DELTA performs comparably in identifying ICU transfer or cardiac arrest with SEN=73.1%, SPEC=98.2% relative to SEN=84%, SPEC=98%. With the 72-hour discount, DELTA outperforms the neural network with SEN=91.3%, SPEC=98%. DELTA is scalable and can analyze the ever-increasing hospitalized and, theoretically, outpatient-and/or-wearable patient data streams. The hope is that earlier recognition of deterioration will prompt earlier interventions and improve patient outcomes.