Background: Resident physicians are responsible for mastering a vast array of clinical and affective competencies. In the modern clinical learning environment, it is vital that residents can effectively use the electronic health record (EHR) to deliver safe and high-quality care, and that attending physicians and health organizations develop standards by which to supervise trainees in doing so. The ways in which hospitalist trainees’ use of the EHR changes during training are poorly understood, and little work has sought to establish measures of learner competence in use of the EHR.
Methods: This project uses EHR audit log data extracted from the University of Minnesota Medical Center Epic EHR for the 6-month period between January and June of 2022. Audit log data is granular metadata that captures how a user interacts with the EHR. Our sample includes all residents scheduled on a hospital medicine service in the study period. We built measures of intensity and nature of EHR use using previously validated activity categories, then used them as inputs to a multi-step clustering algorithm that summarizes each shift as an interpretable sequence of activity states and searches for common patterns of behavior. We applied sequence analysis tools to explore variation in behaviors both longitudinally and by learner level, and to analyze the behavior clusters for signals of improvements in EHR mastery.
Results: 66 residents worked 1116 shifts, comprised of more than 90,000 total 15-minute work periods. The algorithm sequenced the shifts and learned 4 distinct phenotypes of shift-level behavior. We calculated the uncertainty in the state distribution of EHR use for shifts worked by residents of each PG-year. Residents grew more alike in their use of the EHR as they moved through training. PGY-1 residents had the greatest variability, and variability decreased with greater experience. Additionally, our longitudinal analysis of all shifts found residents’ behaviors converged to more predictable patterns over the course of the study. Shifts towards the end of the 2022 training year were markedly more alike than earlier ones, with a modelled decrease of -13.3% (p < 0.001).The learned clusters varied with regards to total time in the EHR as well as time spent writing notes or performing review activities. Despite similar patterns of morning work, the clusters varied with regards to the timing of note-writing and after-shift EHR time. More than 40% of shifts in the least efficient cluster were actively using the EHR at 9PM, compared to only 4.9% of those in the most efficient cluster. Interns were more likely (p = 0.004) to have shifts in less efficient phenotypes than senior residents. Among senior residents, increasing PG-year was associated with higher efficiency (p < 0.001).
Conclusions: Our findings suggest learners vary with regards to their note-writing and overall EHR efficiency, an important understanding in guiding development of measures of EHR competency for learners. We find that these differences are partially attributable to experience. Additionally, we find evidence that learners engage in co-learning as demonstrated by the convergence of EHR use patterns over time. The EHR plays an outsized role in the modern clinical environment, and medical educators must grapple with the challenges of ensuring that trainees can adequately use it to deliver excellent care. Metadata analyses provide an opportunity for rigorous, data-driven, and scalable evaluation of the effective education of residents’ use of this critical tool.
