Background: End of rotation evaluations by attending physicians document resident performance and inform promotion. Unsatisfactory response rates and inadequate qualitative responses are a common problem. To address these issues, we implemented a single-question, Entrustable Professional Activities (EPA) based assessment tool on our inpatient ward rotations and compared it to our existing ACGME milestone-based assessment tool.
Methods: Retrospective cohort of deidentified internal medicine residents at the University of California, San Francisco (UCSF) whose assigned assessments were available through MedHub from January 2020 to December 2021. We compared responses for assigned assessments at our University Hospital (11-question, milestone-based) to our Veterans Affairs and County Hospitals (single question, EPA-based). All residents rotate through all three hospitals. Both assessments used the same overall summary entrustment scores.
Results: Summary assessment scores remained similar between all three sites regardless of assessment type, but 1079/1250 (86%) of EPA assessments were completed, compared to 985/1395 (70%) of milestone assessments (p < 0.05). Descriptive word count data increased in both groups between the two years studied. EPA assessments averaged 111.5 words and milestone evaluations 90.7 words over the two years evaluated although this was not statistically different.
Conclusions: Single-question EPA evaluations have similar overall assessment scores, are associated with significantly higher completion rates, and may demonstrate increased written feedback for our cohort of medical residents. We hope to further assess the qualitative feedback quality between the two assessments in future studies.
