國立虎尾科技大學 |

Generating Faithful and Complete Hospital-Course Summaries From the Electronic Health Record.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Generating Faithful and Complete Hospital-Course Summaries From the Electronic Health Record./
Author:	Adams, Griffin.
Description:	1 online resource (173 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-10, Section: B.
Contained By:	Dissertations Abstracts International85-10B.
Subject:	Bioinformatics. -
Online resource:	click for full text (PQDT)
ISBN:	9798382200217

Generating Faithful and Complete Hospital-Course Summaries From the Electronic Health Record.
Adams, Griffin.

Generating Faithful and Complete Hospital-Course Summaries From the Electronic Health Record. - 1 online resource (173 pages)

Source: Dissertations Abstracts International, Volume: 85-10, Section: B.

Thesis (Ph.D.)--Columbia University, 2024.

Includes bibliographical references

The rapid adoption of Electronic Health Records (EHRs)-electronic versions of a patient's medical history-has been instrumental in streamlining administrative tasks, increasing transparency, and enabling continuity of care across providers. An unintended consequence of the increased documentation burden, however, has been reduced face-time with patients and, concomitantly, a dramatic rise in clinician burnout. Time spent maintaining and making sense of a patient's electronic record is a leading cause of burnout. In this thesis, we pinpoint a particularly time-intensive, yet critical, documentation task: generating a summary of a patient's hospital admissions, and propose and evaluate automated solutions. In particular, we focus on faithfulness, i.e., accurately representing the patient record, and completeness, i.e., representing the full context, as the sine qua non for safe deployment of a hospital-course summarization tool in a clinical setting.The bulk of this thesis is broken up into four chapters: §2 Creating and Analyzing the Data, §3 Improving the Faithfulness of Summaries, §4 Measuring the Faithfulness of Summaries, and, finally, §5 Generating Grounded, Complete Summaries with LLMs. Each chapter links back to the core themes of faithfulness and completeness, while the chapters are linked to each other in that the findings from each chapter shape the direction of subsequent chapters.Given the documentation authored throughout a patient's hospitalization, hospital-course summarization requires generating a lengthy paragraph that tells the story of the patient admission. In §2, we construct a dataset based on 109,000 hospitalizations (2M source notes) and perform exploratory analyses to motivate future work on modeling and evaluation (Adams et al. 2021) [NAACL 2021]. The presence of highly abstractive, entity dense references, coupled with the high stakes nature of text generation in a clinical setting, motivates us to focus on faithfulness and adequate coverage of salient medical entities.In §3, we address faithfulness from a modeling perspective by revising noisy references (Adams et al. 2022) [EMNLP 2022] and, to reduce the reliance on references, directly calibrating model outputs to metrics (Adams et al. 2023c) [ACL 2023].These works relied heavily on automatic metrics as human annotations were limited. To fill this gap, in §4, we conduct a fine-grained expert annotation of system errors in order to meta-evaluate existing metrics and better understand task-specific issues of domain adaptation and source-summary alignments. We find that automatically generated summaries can exhibit many errors, including incorrect claims and critical omissions, despite being highly extractive. These errors are missed by existing metrics. To learn a metric which is less correlated to extractiveness (copy-and-paste), we derive noisy faithfulness labels from an ensemble of existing metrics (Adams et al. 2023d) and train a faithfulness classifier on these pseudo labels [MLHC 2023].Finally, in §5, we demonstrate that fine-tuned LLMs (Mistral (Jiang et al. 2023) and Zephyr (Tunstall et al. 2023)) are highly prone to entity hallucinations and cover fewer salient entities. We improve both coverage and faithfulness by performing sentence-level entity planning based on a set of pre-computed salient entities from the source text, which extends our work on entity-guided news summarization ((Adams et al. 2023a) [ACL, 2023], (Adams et al. 2023b) [EMNLP, 2023].

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798382200217Subjects--Topical Terms:

583857
Bioinformatics.
Subjects--Index Terms:

Large language modelsIndex Terms--Genre/Form:

554714
Electronic books.

Generating Faithful and Complete Hospital-Course Summaries From the Electronic Health Record.
LDR:04877ntm a22003857 4500 001 1151490
005 20241112095336.5
006 m o d
007 cr mn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798382200217
035 $a (MiAaPQ)AAI30995197
035 $a AAI30995197
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Adams, Griffin. $3 1478268
245 1 0 $a Generating Faithful and Complete Hospital-Course Summaries From the Electronic Health Record.
264 0 $c 2024
300 $a 1 online resource (173 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-10, Section: B.
500 $a Advisor: Elhadad, Noemie.
502 $a Thesis (Ph.D.)--Columbia University, 2024.
504 $a Includes bibliographical references
520 $a The rapid adoption of Electronic Health Records (EHRs)-electronic versions of a patient's medical history-has been instrumental in streamlining administrative tasks, increasing transparency, and enabling continuity of care across providers. An unintended consequence of the increased documentation burden, however, has been reduced face-time with patients and, concomitantly, a dramatic rise in clinician burnout. Time spent maintaining and making sense of a patient's electronic record is a leading cause of burnout. In this thesis, we pinpoint a particularly time-intensive, yet critical, documentation task: generating a summary of a patient's hospital admissions, and propose and evaluate automated solutions. In particular, we focus on faithfulness, i.e., accurately representing the patient record, and completeness, i.e., representing the full context, as the sine qua non for safe deployment of a hospital-course summarization tool in a clinical setting.The bulk of this thesis is broken up into four chapters: §2 Creating and Analyzing the Data, §3 Improving the Faithfulness of Summaries, §4 Measuring the Faithfulness of Summaries, and, finally, §5 Generating Grounded, Complete Summaries with LLMs. Each chapter links back to the core themes of faithfulness and completeness, while the chapters are linked to each other in that the findings from each chapter shape the direction of subsequent chapters.Given the documentation authored throughout a patient's hospitalization, hospital-course summarization requires generating a lengthy paragraph that tells the story of the patient admission. In §2, we construct a dataset based on 109,000 hospitalizations (2M source notes) and perform exploratory analyses to motivate future work on modeling and evaluation (Adams et al. 2021) [NAACL 2021]. The presence of highly abstractive, entity dense references, coupled with the high stakes nature of text generation in a clinical setting, motivates us to focus on faithfulness and adequate coverage of salient medical entities.In §3, we address faithfulness from a modeling perspective by revising noisy references (Adams et al. 2022) [EMNLP 2022] and, to reduce the reliance on references, directly calibrating model outputs to metrics (Adams et al. 2023c) [ACL 2023].These works relied heavily on automatic metrics as human annotations were limited. To fill this gap, in §4, we conduct a fine-grained expert annotation of system errors in order to meta-evaluate existing metrics and better understand task-specific issues of domain adaptation and source-summary alignments. We find that automatically generated summaries can exhibit many errors, including incorrect claims and critical omissions, despite being highly extractive. These errors are missed by existing metrics. To learn a metric which is less correlated to extractiveness (copy-and-paste), we derive noisy faithfulness labels from an ensemble of existing metrics (Adams et al. 2023d) and train a faithfulness classifier on these pseudo labels [MLHC 2023].Finally, in §5, we demonstrate that fine-tuned LLMs (Mistral (Jiang et al. 2023) and Zephyr (Tunstall et al. 2023)) are highly prone to entity hallucinations and cover fewer salient entities. We improve both coverage and faithfulness by performing sentence-level entity planning based on a set of pre-computed salient entities from the source text, which extends our work on entity-guided news summarization ((Adams et al. 2023a) [ACL, 2023], (Adams et al. 2023b) [EMNLP, 2023].
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Bioinformatics. $3 583857
650 4 $a Medicine. $3 644133
653 $a Large language models
653 $a Summarization
653 $a Electronic Health Records
653 $a Medical history
655 7 $a Electronic books. $2 local $3 554714
690 $a 0715
690 $a 0564
690 $a 0769
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a Columbia University. $b Computer Science. $3 1179509
773 0 $t Dissertations Abstracts International $g 85-10B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30995197 $z click for full text (PQDT)