Background: Feedback in clinical training is essential to reinforce positive behaviors and to provide areas for improvement. Numeric ratings from student performance evaluations are helpful but are not sufficient to guide the learner about where they may be excelling and where they may need additional development. Narrative written feedback both reinforces verbal feedback and provides support for any numeric ratings. Narrative feedback on clinical evaluations is also vital so medical schools have a record of that specific feedback. Medical schools then can use that feedback to guide coaching and remediation plans, if needed. In addition, these narrative comments play an important role on the medical student performance evaluation letter to pass on meaningful student information to prospective residency programs.
Methods: We designed a pre-post study to evaluate the impact of general and targeted faculty development on narrative feedback for two hospital medicine groups in their evaluations of students on the Internal Medicine clerkship. We found published narrative rubrics overly complex or lacking in key elements, so we created a rubric with three sections (positive feedback, constructive feedback, and action to promote improvement), each on a 0-1-2 scale (Table 1). To assess interrater reliability, three authors blindly scored 146 evaluations from 10 faculty in 2018-2019 with an intraclass correlation of 0.959. For the baseline data, we blindly scored Internal Medicine clerkship evaluations during the 2019-2020 year. With this data, we delivered a large group didactic, emphasizing the best practices of narrative feedback to both groups (general feedback). During the 2020-2021 year, we met one-on-one with each hospitalist to review their performance compared to peers (targeted feedback). After the 2021-2022 year, we again blindly scored faculty evaluations. Paired t-tests were used to compare means.
Results: We included 24 hospitalists, excluding those who left the institution or completed ≤ 3 evaluations. For the 2019-2020 baseline, the combined mean score was 3.7/6 (SD = 1.2) compared to 4.6/6 (SD = 1.1) for the 2021-2022 post-intervention period (p = 0.0002). See Figure 1 for combined mean score comparison. Mean scores within each hospital medicine subgroup, combined mean scores for constructive feedback, and mean scores for the action to promote improvement for each subgroup were all also improved with p values < 0.05. As potential limitations, this study was at two sites that could limit its extrapolation. Though the authors were blinded, confirmation bias could have influenced the post-intervention scoring.
Conclusions: The use of a simplified narrative feedback rubric with a combination of general and targeted faculty development can improve narrative feedback from faculty to medical students.
