Background: Clinical handbooks support patient care, with studies demonstrating positive effects in areas such as quality of chart documentation, clinical knowledge, and value of inpatient care. Internal medicine residents rotating through inpatient services are particularly likely to benefit from handbooks, especially when handbooks share local, system-specific, and practical guidance. However, less is known about how to improve the quality of handbooks’ contents through direct observation of readers’ learning habits.
Purpose: Under hospitalist leadership, internal medicine residents at a large academic medical center created a handbook to guide management of common clinical problems. Each year, residents revise and update the handbook, before revisions undergo review by faculty, with more than 201 authors and reviewers contributing over five editions. The handbook is updated and published online at VIMBook.org, where it is freely available for any reader to access. To improve our approach to revisions to the handbook, we used data from Google Analytics, a service that reports metrics related to website traffic, to learn more about the behavior of readers.
Description: Over twelve months (10/2022 – 10/2023), most visits (>45,000) to the site were from cities across the US, with the remainder (~10,000) from other countries. Visits were split nearly evenly between mobile and desktop (27,727 vs 27,058, with 482 on tablets). Figure 1 shows the change in how users were referred to the handbook site, with more arriving via internet search over time. Table 1 lists the sections that readers consulted most often. Notably, these sections are succinct, providing bulleted guidance that readers checked rapidly. The majority of readers spent very little time on these pages, often less than one minute.
Conclusions: Historically, studies of preferences for learning and answering clinical questions solicit feedback from the readers of handbooks, rather than drawing upon observations of their behavior. Our group has conducted similar qualitative research, but we have found that pairing qualitative and quantitative metrics allows a deeper understanding of readers’ preferences. Thus far, our data show that learners gravitate very strongly towards detailed but rapidly attained guidance, often accessed via a general internet search and not from direct consultation with a preferred resource. Because approximately half of our visits are from direct referral (and not internet search) we also infer that users are still finding the information they seek, rather than simply navigating to the site via search, without eventually finding answers to their questions. A review of the most popular pages suggests that succinct (even bulleted) prose is preferred, which is in line with the observation that users spend relatively short periods of time on pages.
