Background: Educational efforts targeting diagnostic reasoning have been proposed by the National Academy of Medicine as a means to reduce diagnostic errors. Building on situativity theory, which describes how the context in which we reason impacts our diagnostic ability, teaching diagnostic reasoning should ideally occur in tandem with patient care. Tracking patient outcomes can improve diagnostic reasoning by providing performance feedback and enabling iterative calibration in the context of a busy workplace and a diverse patient population. Although prior work shows that internal medicine residents endorse these practices, they do not consistently track outcomes for patients they admit overnight. Given the focus on admitting undifferentiated patients, the night float rotation provides an ideal setting for intentional diagnostic reflection.
Purpose: We piloted a novel tool for internal medicine residents to identify working diagnoses and level of certainty upon admission, track diagnostic changes at 72 hours, and reflect on their diagnostic reasoning. The goal of the project was to provide a structured framework to encourage guided diagnostic reflection and calibration.
Description: Second-year residents completing their night float rotation voluntarily used the novel Diagnostic Reflection and Feedback Tool (DRAFT) to track three admissions per night, record the working diagnosis, level of diagnostic certainty, and patient identity categories (including gender presentation, perceived race, weight category, age range, and need for an interpreter). They documented any diagnostic changes after 72 hours and reflected on salient case features and/or contextual factors that either supported a correct working diagnosis or played a key role in the diagnostic evolution. Investigators reviewed charts and categorized the degree of diagnostic change. We assessed residents’ attitudes and practices around diagnostic reflection using baseline and post-intervention surveys. We used data from DRAFT entries to evaluate the association between residents’ certainty and degree of diagnostic change. IRB approval was obtained from the University of Rochester.
Conclusions: Pre-intervention surveys revealed that although most participating residents track patient outcomes for interesting cases, the lack of a reliable system hinders the process. Further, the majority of residents were only “somewhat” or “not at all” confident in their diagnostic accuracy, highlighting the importance of educational efforts targeting diagnostic reasoning. Of 24 residents who completed the second-year night float rotation, 10 accessed the DRAFT and logged a total of 138 cases. Although residents’ certainty was inversely correlated with the degree of diagnostic change, there was a notable calibration gap: 6.5% of high certainty cases had a major diagnostic change and 35% of low certainty cases had no diagnostic change. Our novel tool facilitates diagnostic reflection for cases with varying degrees of certainty, allowing users to calibrate their reasoning and “catch” seemingly straightforward cases that evolve. Study limitations includes its voluntary nature which can introduce selection bias and inhibit broader resident participation. Future work involves building the DRAFT into the electronic medical record to integrate the reflection process into the workflow of busy residents. We are also working with residency leadership to include intentional diagnostic reflection as a formal component of residency education.