Background: Feedback in clinical training is essential to reinforce positive behaviors and to provide areas for improvement. Numeric ratings from student performance evaluations are helpful but are not sufficient to guide the learner in areas where they excel and where they may need additional development. Narrative written feedback both reinforces verbal feedback and provides support for any numeric ratings. Medical schools can use that narrative feedback to guide coaching and remediation plans, if needed. In addition, these comments play an important role on the Medical Student Performance Evaluation letter to pass on meaningful student information to prospective residency programs.Despite these benefits, it can be challenging to get faculty to write meaningful narrative feedback. First, they may not always recognize what meaningful feedback looks like. Second, faculty are pulled in more directions than ever and have limited time to devote to written evaluations. It is crucial to ensure faculty have the skills to write concise narrative feedback that is specific, behavioral-based, and balanced.
Purpose: Our goal was to improve the quality of written feedback by hospitalists to third-year medical students through an educational quality improvement project.
Description: Through a literature review and consensus, we first identified the key principals of meaningful narrative feedback. This ultimately included specific comments that were 1) positive, 2) constructive and 3) included actions to promote improvement. To help quantify narrative feedback, we reviewed the literature and found narrative feedback rubrics that were deemed overly complex for our needs. Through iterative revisions, we then created a simplified narrative feedback rubric (Figure 1).To assess reliability of the rubric, three authors were blinded and used the rubric to each score 146 third-year student evaluations from 10 faculty from the 2018-2019 year (pre-study period). The results showed high interrater reliability (intraclass correlation coefficient = 0.959).For the 2019-2020 academic year (pre-intervention period), three authors were blinded and scored all third-year student evaluations from all faculty in hospital medicine in our two groups (University and VA) who completed a minimum of three student evaluations (34 total faculty, 323 evaluations).During the 2020-2021 academic year, we introduced the project and promoted improvement through a workshop on meaningful narrative feedback for each group. The workshop stressed the importance of narrative feedback, examples of high-quality narrative feedback, and strategies to compose effective narrative feedback. We displayed average scores for each group using our written feedback rubric. Next, the scores from 2019-2020 were unblinded, and we had one-on-one meetings with individual faculty to review each faculty’s standardized score report (Figure 2), which included the faculty’s average score for each component with a histogram to compare their overall average score to their peers. We provided specific positive and actionable constructive feedback for each faculty member.After the 2021-2022 academic year (post-intervention period), we will score evaluations from the same faculty members and assess for improvement.
Conclusions: By combining both general and targeted faculty development with a simplified and less time intensive scoring rubric, we hope to improve written feedback for our students.
