國立虎尾科技大學 |

Automatic Speech Separation for Brain-Controlled Hearing Technologies.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Automatic Speech Separation for Brain-Controlled Hearing Technologies./
Author:	Han, Cong.
Description:	1 online resource (174 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-07, Section: B.
Contained By:	Dissertations Abstracts International85-07B.
Subject:	Electrical engineering. -
Online resource:	click for full text (PQDT)
ISBN:	9798381445657

Automatic Speech Separation for Brain-Controlled Hearing Technologies.
Han, Cong.

Automatic Speech Separation for Brain-Controlled Hearing Technologies. - 1 online resource (174 pages)

Source: Dissertations Abstracts International, Volume: 85-07, Section: B.

Thesis (Ph.D.)--Columbia University, 2024.

Includes bibliographical references

Speech perception in crowded acoustic environments is particularly challenging for hearing impaired listeners. While assistive hearing devices can suppress background noises distinct from speech, they struggle to lower interfering speakers without knowing the speaker on which the listener is focusing. The human brain has a remarkable ability to pick out individual voices in a noisy environment like a crowded restaurant or a busy city street. This inspires the brain-controlled hearing technologies. A brain-controlled hearing aid acts as an intelligent filter, reading wearers' brainwaves and enhancing the voice they want to focus on. Two essential elements form the core of brain-controlled hearing aids: automatic speech separation (SS), which isolates individual speakers from mixed audio in an acoustic scene, and auditory attention decoding (AAD) in which the brainwaves of listeners are compared with separated speakers to determine the attended one, which can then be amplified to facilitate hearing. This dissertation focuses on speech separation and its integration with AAD, aiming to propel the evolution of brain-controlled hearing technologies. The goal is to help users to engage in conversations with people around them seamlessly and efficiently.This dissertation is structured into two parts. The first part focuses on automatic speech separation models, beginning with the introduction of a real-time monaural speech separation model, followed by more advanced real-time binaural speech separation models. The binaural models use both spectral and spatial features to separate speakers and are more robust to noise and reverberation. Beyond performing speech separation, the binaural models preserve the interaural cues of separated sound sources, which is a significant step towards immersive augmented hearing. Additionally, the first part explores using speaker identifications to improve the performance and robustness of models in long-form speech separation. This part also delves into unsupervised learning methods for multi-channel speech separation, aiming to improve the models' ability to generalize to real-world audio. The second part of the dissertation integrates speech separation introduced in the first part with auditory attention decoding (SS-AAD) to develop brain-controlled augmented hearing systems. It is demonstrated that auditory attention decoding with automatically separated speakers is as accurate and fast as using clean speech sounds. Furthermore, to better align the experimental environment of SS-AAD systems with real-life scenarios, the second part introduces a new AAD task that closely simulates real-world complex acoustic settings. The results show that the SS-AAD system is capable of improving speech intelligibility and facilitating tracking of the attended speaker in realistic acoustic environments. Finally, this part presents employing self-supervised learned speech representation in the SS-AAD systems to enhance the neural decoding of attentional selection.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798381445657Subjects--Topical Terms:

596380
Electrical engineering.
Subjects--Index Terms:

Auditory attention decodingIndex Terms--Genre/Form:

554714
Electronic books.

Automatic Speech Separation for Brain-Controlled Hearing Technologies.
LDR:04384ntm a22003857 4500 001 1150135
005 20241022111555.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798381445657
035 $a (MiAaPQ)AAI30990427
035 $a AAI30990427
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Han, Cong. $3 1476566
245 1 0 $a Automatic Speech Separation for Brain-Controlled Hearing Technologies.
264 0 $c 2024
300 $a 1 online resource (174 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-07, Section: B.
500 $a Advisor: Mesgarani, Nima.
502 $a Thesis (Ph.D.)--Columbia University, 2024.
504 $a Includes bibliographical references
520 $a Speech perception in crowded acoustic environments is particularly challenging for hearing impaired listeners. While assistive hearing devices can suppress background noises distinct from speech, they struggle to lower interfering speakers without knowing the speaker on which the listener is focusing. The human brain has a remarkable ability to pick out individual voices in a noisy environment like a crowded restaurant or a busy city street. This inspires the brain-controlled hearing technologies. A brain-controlled hearing aid acts as an intelligent filter, reading wearers' brainwaves and enhancing the voice they want to focus on. Two essential elements form the core of brain-controlled hearing aids: automatic speech separation (SS), which isolates individual speakers from mixed audio in an acoustic scene, and auditory attention decoding (AAD) in which the brainwaves of listeners are compared with separated speakers to determine the attended one, which can then be amplified to facilitate hearing. This dissertation focuses on speech separation and its integration with AAD, aiming to propel the evolution of brain-controlled hearing technologies. The goal is to help users to engage in conversations with people around them seamlessly and efficiently.This dissertation is structured into two parts. The first part focuses on automatic speech separation models, beginning with the introduction of a real-time monaural speech separation model, followed by more advanced real-time binaural speech separation models. The binaural models use both spectral and spatial features to separate speakers and are more robust to noise and reverberation. Beyond performing speech separation, the binaural models preserve the interaural cues of separated sound sources, which is a significant step towards immersive augmented hearing. Additionally, the first part explores using speaker identifications to improve the performance and robustness of models in long-form speech separation. This part also delves into unsupervised learning methods for multi-channel speech separation, aiming to improve the models' ability to generalize to real-world audio. The second part of the dissertation integrates speech separation introduced in the first part with auditory attention decoding (SS-AAD) to develop brain-controlled augmented hearing systems. It is demonstrated that auditory attention decoding with automatically separated speakers is as accurate and fast as using clean speech sounds. Furthermore, to better align the experimental environment of SS-AAD systems with real-life scenarios, the second part introduces a new AAD task that closely simulates real-world complex acoustic settings. The results show that the SS-AAD system is capable of improving speech intelligibility and facilitating tracking of the attended speaker in realistic acoustic environments. Finally, this part presents employing self-supervised learned speech representation in the SS-AAD systems to enhance the neural decoding of attentional selection.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Electrical engineering. $3 596380
650 4 $a Speech therapy. $3 715798
650 4 $a Audiology. $3 889171
653 $a Auditory attention decoding
653 $a Brain-computer interface
653 $a Smart hearing aid
653 $a Speech separation
655 7 $a Electronic books. $2 local $3 554714
690 $a 0544
690 $a 0300
690 $a 0460
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a Columbia University. $b Electrical Engineering. $3 1179181
773 0 $t Dissertations Abstracts International $g 85-07B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30990427 $z click for full text (PQDT)