Background: Sepsis is a major cause of mortality in hospitalized patients, and early treatment is critical to survival. However, there is a paucity of research on methods to assist real-time clinical decision-making for sepsis treatment. Utilizing data on patients who had sepsis in the ICU, we propose a novel data-driven framework for recommending treatments by grouping patients into representative meta-states and find the optimal treatment for each meta-state.
Methods: The sepsis treatment problem is modeled as a Markov Decision Process (MDP). At each decision point, a clinician decides which action to take (e.g., starting IV fluids, vasopressors, antibiotics etc.) based on the clinical status of the patient. Presenting patients have diverse characteristics, which could signify many possible distinct “health states”. Instead of finding a recommended action for patient characteristic combination (an intractable task), we use a novel approach to aggregate the state space into a small number of representative meta-states, and find the optimal action for each meta-state. These meta-states are able to capture the dynamics of the original system, i.e. the optimal actions for each patient remain the same.Patients are grouped into meta-states based on clinical status, severity of sepsis (measured by SOFA score) as well as transition probabilities to other meta-states (capturing “similar reactions” to the same treatment). After the original feature space has been aggregated into meta-states, we use context-appropriate algorithms to find recommended action(s) for each meta-state, optimizing for a pre-specified outcome (such as minimizing SOFA score, incentivizing discharge, disincentivizing readmission and death).The final meta-states are validated in collaboration with clinicians. We derive an interpretable model for assigning patients to meta-states – a decision tree that asks a series of questions about a patient to determine the meta-state in which they fall. We visualize the defining features of each meta-state using a variety of plots and graphical analyses. Finally, we conduct a simulation on a held-out test set, taking the actions recommended by the algorithm. We compare predicted outcomes such as length of stay, number of patients discharged from the ICU, readmitted, deceased, and average SOFA score against the observed outcomes in the data on the same cohort.
Results: Our training dataset consists of 11K sepsis patients treated in the ICU at a major Northeastern US Hospital. Our algorithm aggregated the original state space into 24 meta-states, which are defined by characteristics such as SOFA score, length of stay in the ICU, current treatments, and vital signs.A simulation study of the predicted trajectories of 2K patients in the held-out test set concludes that, when following our recommended policies for each meta-state, the number of discharges increases by at least at 7%, the number of deaths decreases by at least 13%, the length of stay decreases by at least 50%, and the average SOFA score decreases by at least 10%.
Conclusions: We provide a data-driven framework that utilizes novel algorithms to make sepsis treatment recommendations by grouping patients into meta-states and solving an MDP. The method shows potential to reduce the number of deaths, increase the number of discharges, and reduce the length of stay of sepsis patients in the ICU. Our ongoing work includes further clinical validation with expanded patient data consisting of more clinical features, treatments and specific outcomes.