Background: Traditional measures of workload such as wRVUs may not be adequate to understand the impact of work design on key patient, workforce, and organizational outcomes. In this study, we developed a workforce application to capture clinician perception of workload and environment and then paired this data with electronic workload measures in the EHR in order to gain a more comprehensive understanding of clinician work experience through both objective and subjective measures.
Methods: We developed the GrittyWork application (GW App) using the Chokshi and Mann Process Model for User-Centered Digital Development at a 678-bed quaternary care hospital. The aim of the GW App is to capture, in real-time, measures of clinician workload and perception of work environment. Leveraging Ecological Momentary Assessment methods, we crafted a data acquisition process to allow for real-time data collection over time and integrated processes for extracting electronic health record (EHR) based workload measures at a single academic medical center. Participants included hospitalist physicians and advanced practice providers (APPs). Measures included the System Usability Scale (SUS) (a 10-item questionnaire with the highest possible score of 100 with a higher score indicating better usability), use metrics from GW App, audit log data (workload data automatically captured via clinician use of the EHR during regular daily work), EHR note counts, and qualitative interviews.
Results: From October 28, 2022 to November 3, 2022, six hospitalist clinicians provided feedback on context of use, user requirements, and the early prototype of the GW App. From February 28, 2023 to June 8, 2023, thirty hospitalist clinicians participated in the pilot while on clinical service. All 30 clinicians (100%) participated in the pilot submitting data for a total of 122 shifts. Twenty-three (77%) of participants were physicians and 7 (23%) were APPs. Sixty percent of participants were women. Participants on average were 6.5 ± 4.5 years since training and had an average clinical effort of 74.5% ± 24.1%. On average participants reported working 10 + 1 hours per day and were responsible for an average of 11 + 3 patients per day. The post-pilot evaluation of the GW App showed a SUS score of 86 + 11 (mean + SD) and a participant preference towards application-based surveys (73% of participants). In regards to workload measures, EHR audit log data and notes data correlated with physician-reported workloads. However, there were discrepancies between hours worked as reported in the GW App and time recorded in the audit log data for advanced practice providers.
Conclusions: The GW App was successfully used to gather real-time clinician experiences. These findings were paired with EHR audit log data to provide insights into workloads and work patterns paving the way for future work to connect these measures with specific outcomes to identify when work design may lead to harm.
