Background:
Hand‐offs are a core competency of hospitalists. Although the Society of Hospital Medicine and others recommend improving hand‐offs, monitoring and improving hand‐off quality are limited by lack of reliable tools to measure hand‐off quality. This study aimed to assess the feasibility and reliability of using a paper‐based tool, “Handoff CEX (Clinical Evaluation Exercise),” to evaluate real patient hand‐offs between hospitalist clinicians.
Methods:
The Handoff CEX, developed based on literature review and expert consensus, includes ratings of overall performance and its components (organization, communication skills, clinical judgment, setting, patient‐focused) on a 0–9 scale. For 3 hand‐offs a week (Monday/Tuesday/Friday), clinician senders and receivers were evaluated by a trained third‐party nonmedical observer using the Handoff CEX. Senders and receivers also evaluated each other using the instrument. Interrater reliability between clinician and observer was calculated using Spearman's rho. Descriptive and comparative statistics were used to examine mean performance and “superior” performance, defined as the top quartile.
Results:
From March to December 2010, all 38 (100%) hospitalist clinicians (nurse practitioners, hospitalists) consented to participate. Senders, receivers, and a trained observer rated 78 hand‐offs, resulting in 156 participant and 153 observer evaluations. Domain means were between 6 and 7, with full use of the 0–9 scale noted. Internal consistency was high (Cronbach's alpha = 0.90). Spearman's rho between participating clinicians and trained observer was calculated as 0.52 (P < 0.001), indicating moderate interrater reliability. Although tardiness was noted in only 9% of hand‐offs, nearly all ratings were lower if a clinician arrived late (overall, 7.26 not tardy vs. 5.85 tardy, P < 0.001). Setting was rated significantly higher on Monday than on other days (7.50 Monday vs. 6.75 Tuesday/Friday, P < 0.001). Clinician senders (starting shift) were significantly less likely to provide superior (top quartile) ratings in 3 areas (overall, organization, setting) than were receivers (ending shift). Observer ratings did not show this disparity. Evaluator satisfaction with the tool was high (mean, 6.80; IQR, 6–8) and was also associated with overall hand‐off quality (β = 0.60, P < 0.001).
Conclusions:
Real‐time assessment of hand‐off quality by clinicians using the Handoff CEX is feasible and reliable. Arriving late to hand‐offs can dramatically affect ratings of hand‐off quality. Other characteristics, such as day of week and sender/receiver roles, are also related to hand‐off ratings. It may be easier to critically evaluate senders, who bear the burden of communication, than receivers. Alternatively, receivers may be more critical because of the stress of receiving work, or senders may overestimate receiver performance because of the excitement of ending their shift. Further work to explore the mechanism of these findings is under way.
Disclosures:
V. Arora ‐ AHRQ, NIA, ABIM, ACGME, research funding; P. Staisiunas ‐ AHRQ, research funding; S. Banerjee ‐ none; E. Greenstein ‐ NIA, research funding; L. Horwitz ‐ NIA, AHRQ, research funding; J. Farnan ‐ AHRQ, research funding