Background: As patient satisfaction surveys become a routine part of physician performance evaluations, it is increasingly important to study the tools that are utilized to evaluate this performance. Little research has been performed to assess the metrics of these tools. Top box percentage is frequently used to report physician performance but the relationship of top box measurements with more traditional measures such as mean scores has not been thoroughly evaluated. The purpose of our study was to evaluate the results of physician evaluations when scores are calculated using the top box methodology as compared with the mean score.
Methods: We completed this prospective observational study at a suburban voluntary hospital that has 318 inpatient beds. All patients admitted to the telemetry floor were eligible. We excluded patients who were unable or refused to answer the questionnaire. The anonymous questionnaire was administered over a 4-month period by a team of trained research assistants who asked the patient to identify their physician by name as well as answer 5 additional questions designed to assess: physician communication, concern, introduction to the patient, answering patient questions and whether the patient would recommend the physician. Patient satisfaction was rated on 3-point scale.
Results: During the 3-month period of the study 158 patients evaluated 15 hospitalists yielding an average of 9.9 evaluations per hospitalist. Hospitalists who did not have at least 5 evaluations were excluded for the analysis. A question specific mean score and top box percent score were calculated for each hospitalist. The top box score was the percentage of evaluations that gave the highest score for the specific question. We calculated a correlation coefficient between the individual physicians’ mean score and the top box score for each question on the survey. The top box score was highly correlated with the mean score across all questions (Q1 r= .94, p=.0001, Q2 r= .97, p=.0001, Q3 r= .96, p=.0001, Q4 r= .96, p=.0001, Q5 r= .92, p=.0001).
Conclusions: The measurement of hospitalist performance on a patient satisfaction utilizing the top box percent was very highly correlated with results when the mean score was calculated. Our study suggests that the top box percent score is essentially interchangeable with the mean score.