國立虎尾科技大學 |

End-to-End Modeling for Abstractive Speech Summarization.

紀錄類型:	書目-語言資料,手稿 : Monograph/item
正題名/作者:	End-to-End Modeling for Abstractive Speech Summarization./
作者:	Sharma, Roshan.
面頁冊數:	1 online resource (130 pages)
附註:	Source: Dissertations Abstracts International, Volume: 85-09, Section: A.
Contained By:	Dissertations Abstracts International85-09A.
標題:	Communication. -
電子資源:	click for full text (PQDT)
ISBN:	9798381953312

End-to-End Modeling for Abstractive Speech Summarization.
Sharma, Roshan.

End-to-End Modeling for Abstractive Speech Summarization. - 1 online resource (130 pages)

Source: Dissertations Abstracts International, Volume: 85-09, Section: A.

Thesis (Ph.D.)--Carnegie Mellon University, 2024.

Includes bibliographical references

In our increasingly interconnected world, where speech remains the most intuitive and natural form of communication, spoken language processing systems face a crucial challenge: they must do more than just categorize speech, they need to truly understand it to generate meaningful responses. One key aspect of this understanding is speech summarization, where a system condenses the important information from spoken input into a concise summary. This thesis delves into the challenge of generating abstractive textual summaries directly from speech.The classical approach involves cascade systems that realize speech summarization by first transcribing speech, and then summarizing the resulting transcript. However, this comes with many challenges including computational efficiency, domain mismatches, and error propagation. In this thesis, we propose an alternative-an end-to-end framework that directly optimizes a single sequence model for speech summarization. To implement such end-to-end models with constrained computing resources, we address challenges such as abstract learning, learning global acoustic context, dealing with paucity of data, and improving the quality of summaries using multiple references. We also shed light on observations from human annotation for speech summarization.We present multi-stage training using speech transcription as a pre-training task to address abstract learning and facilitate improved performance of end-to-end models. We describe multiple solutions to address the problem of global acoustic context-restricted self-attention, replacing self-attention with the Fourier transform, and two block-wise adaptation solutions BASS and R-BASS that reframe speech summarization through the lens of block-wise processing. To address the challenge of data paucity, we introduce work on two new datasets-SLUE-TED and Interview for abstractive speech summarization. An exploration of human annotation provides insights into best practices and the nature of differences between speech-based and transcript-based summaries. Finally, we propose a novel method called AugSumm to improve the diversity and fluency of speech summaries by leveraging auxiliary references from generative text models.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798381953312Subjects--Topical Terms:

556422
Communication.
Subjects--Index Terms:

Abstractive speech summarizationIndex Terms--Genre/Form:

554714
Electronic books.

End-to-End Modeling for Abstractive Speech Summarization.
LDR:03643ntm a22004097 4500 001 1146451
005 20240812064619.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798381953312
035 $a (MiAaPQ)AAI30993760
035 $a AAI30993760
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Sharma, Roshan. $3 1471840
245 1 0 $a End-to-End Modeling for Abstractive Speech Summarization.
264 0 $c 2024
300 $a 1 online resource (130 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-09, Section: A.
500 $a Advisor: Raj, Bhiksha.
502 $a Thesis (Ph.D.)--Carnegie Mellon University, 2024.
504 $a Includes bibliographical references
520 $a In our increasingly interconnected world, where speech remains the most intuitive and natural form of communication, spoken language processing systems face a crucial challenge: they must do more than just categorize speech, they need to truly understand it to generate meaningful responses. One key aspect of this understanding is speech summarization, where a system condenses the important information from spoken input into a concise summary. This thesis delves into the challenge of generating abstractive textual summaries directly from speech.The classical approach involves cascade systems that realize speech summarization by first transcribing speech, and then summarizing the resulting transcript. However, this comes with many challenges including computational efficiency, domain mismatches, and error propagation. In this thesis, we propose an alternative-an end-to-end framework that directly optimizes a single sequence model for speech summarization. To implement such end-to-end models with constrained computing resources, we address challenges such as abstract learning, learning global acoustic context, dealing with paucity of data, and improving the quality of summaries using multiple references. We also shed light on observations from human annotation for speech summarization.We present multi-stage training using speech transcription as a pre-training task to address abstract learning and facilitate improved performance of end-to-end models. We describe multiple solutions to address the problem of global acoustic context-restricted self-attention, replacing self-attention with the Fourier transform, and two block-wise adaptation solutions BASS and R-BASS that reframe speech summarization through the lens of block-wise processing. To address the challenge of data paucity, we introduce work on two new datasets-SLUE-TED and Interview for abstractive speech summarization. An exploration of human annotation provides insights into best practices and the nature of differences between speech-based and transcript-based summaries. Finally, we propose a novel method called AugSumm to improve the diversity and fluency of speech summaries by leveraging auxiliary references from generative text models.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Communication. $3 556422
650 4 $a Acoustics. $3 670692
650 4 $a Linguistics. $3 557829
650 4 $a Computer science. $3 573171
653 $a Abstractive speech summarization
653 $a Speech recognition
653 $a Human summarization
653 $a Human annotations
653 $a Spoken language processing
655 7 $a Electronic books. $2 local $3 554714
690 $a 0984
690 $a 0459
690 $a 0290
690 $a 0986
710 2 $a Carnegie Mellon University. $b Electrical and Computer Engineering. $3 1182305
710 2 $a ProQuest Information and Learning Co. $3 1178819
773 0 $t Dissertations Abstracts International $g 85-09A.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30993760 $z click for full text (PQDT)