Background: Hospital discharges can be complex for patients, and ensuring patient comprehension of their clinical records and discharge summaries is critical for improving health-related outcomes. However, studies suggest that 88% of discharge instructions are not readable to the population served. In our prior study we demonstrated that generative AI (AI) has the potential to transform discharge summaries into language that is readable and understandable for patients. Now, we conducted a usability study for patients and clinicians to provide feedback on these AI-generated patient friendly discharge summaries to assess patient and provider comprehension and provider usage of the tool.
Methods: This study takes place at a large academic health system with four hospital campuses. We have previously developed a prompt for GPT-4 to convert discharge summaries into a patient-friendly language and format. We also developed tools to integrate this into our EHR. In this usability study from May to August 2024, we conducted ethnographic interviews with 8 patients and 7 providers. Patients reviewed their own patient friendly discharge summaries and providers reviewed GPT-4 outputs in a testing environment with real clinical data. We analyzed transcripts and field notes from these interviews using a modified grounded theory approach.
Results: We conducted 15 interviews (8 patients, 7 providers). Through qualitative analysis, four main themes emerged. (1) Satisfaction with AI-drafted Discharge Summary: Both patients and clinicians expressed satisfaction with the AI-drafted discharge summaries. Most patients found the AI-generated discharge summaries understandable and readable. They appreciated the simplified language, although some felt it was overly simplified and missed important details. Clinicians valued the comprehensiveness of the summaries but identified some missing or unclear information. They found the decision support system helpful but expressed concerns about “alert fatigue.” (2) General Sentiments Regarding AI: Views towards AI were mixed. Some patients felt uncomfortable with the general idea of AI, while others were not worried and thought AI might be helpful. Clinicians reported general comfort with AI, regardless of self-reported level of expertise regarding AI. (3) Transparency and Disclosure: Patients did not report strong feelings about the idea of a statement disclosing the use of AI to patients. Some clinicians worried that disclosing the use of AI may negatively impact the patient-clinician relationship. (4) Responsibility for Accuracy, Safety, and Privacy: Most patients wanted assurance that their doctors would review and approve AI-generated material. Clinicians reported that they would consistently edit the AI-drafted summaries to personalize them, fix any errors, and add details as necessary.
Conclusions: AI provides an excellent opportunity to improve clinical documentation and enhance patient-provider communication. This is first known study to assess the usability of GPT-4 generated patient discharge summaries, and our initial findings suggest that these summaries are generally helpful and improve clinician workflows but can also miss or misrepresent some information, highlighting the importance of clinician review. These results underscore the potential of generative AI to enhance care transitions and empower patients. Future work should explore scalability and integration into diverse healthcare settings to maximize impact.
