國立虎尾科技大學 |

Towards Inclusive Low-Resource Speech Technologies : = A Case Study of Educational Systems for African American English-Speaking Children.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Towards Inclusive Low-Resource Speech Technologies :/
Reminder of title:	A Case Study of Educational Systems for African American English-Speaking Children.
Author:	Johnson, Alexander.
Description:	1 online resource (119 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-09, Section: A.
Contained By:	Dissertations Abstracts International85-09A.
Subject:	Electrical engineering. -
Online resource:	click for full text (PQDT)
ISBN:	9798381953404

Towards Inclusive Low-Resource Speech Technologies : = A Case Study of Educational Systems for African American English-Speaking Children.
Johnson, Alexander.

Towards Inclusive Low-Resource Speech Technologies :A Case Study of Educational Systems for African American English-Speaking Children. - 1 online resource (119 pages)

Source: Dissertations Abstracts International, Volume: 85-09, Section: A.

Thesis (Ph.D.)--University of California, Los Angeles, 2024.

Includes bibliographical references

The potential of speech technology to improve educational outcomes has been a topic of great interest in recent years. For example, automatic speech recognition (ASR) systems could be employed to provide kindergarten-aged children with real-time feedback on their literacy and pronunciation as they practice reading aloud. Within these systems, speaker identification (SID) technology could additionally be used to identify the user's speaker characteristics in order to ensure that they receive age, language, and dialect-appropriate feedback. While these technologies are more established for well-represented groups in STEM (ie. able-bodied, adult, first-language speakers of mainstream dialects), they give much worse performance for underrepresented groups (young children, speakers of non-mainstream dialects, people with speech-related disabilities, etc.). This work focuses on improving speech technology performance for children's speech and African American English (AAE) dialect speech with the goal of creating more equitable outcomes in early education. The contributions of this work span three primary areas: 1) Dialect identification and density scoring, 2) data augmentation for speech recognition, and 3) Natural Language Processing for fair and inclusive automatic speech assessment.First, we create a robust system for dialect identification of African American English for both children and adult's speech. This system aims to take an input utterance from a speaker of either African American English or Mainstream American English and determine which of the two dialects the utterance belongs. The system fuses features from paralinguistics, self-supervised learning representations, automatic speech recognition system outputs, prosodic contours, and other descriptors of the speech signal in order to learn a mapping from the input acoustic information to a dialect classification decision. We further explore this architecture in automatic dialect density estimation, a task we create and develop. In dialect density scoring, we train a system to automatically predict a speaker's frequency of usage of dialect-specific patterns. This information can then be passed to a speech recognition system for more dialect-informed processing.Second, we develop a data augmentation algorithm to improve zero-shot and few-shot speech recognition of low-resource dialects. The algorithm, named LPCAugment, deconstructs an input speech signal into a source and filter representation using linear predictive coding (LPC) analysis. The poles of the filter representation can then be perturbed independently of the source representation in order to model formant shifts that may be seen across accents and dialects. We use this perturbation method to artificially generate speech samples with shifted formant locations to serve as additional training data for a speech recognition system. This speech recognition system is then evaluated on children's speech for child speakers of a Southern California dialect and child speakers of an Atlanta, Georgia, area dialect.Third, we explore automatic analysis and scoring of speech recognition transcripts for educational assessments. Given information about a student's spoken dialect and automatically generated transcripts of their oral response to an assessment prompt, we train a system to automatically grade the quality of the response with respect to a pre-determined criterion. This system uses language modeling and spoken information retrieval to identify key features in the spoken response and holistically decide if the response aligns with the grading criteria. Combined, the steps in this work form a framework for inclusive spoken language understanding technology that can be used to perform provide students with dialect-appropriate language training or language assessment.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798381953404Subjects--Topical Terms:

596380
Electrical engineering.
Subjects--Index Terms:

Speech technologyIndex Terms--Genre/Form:

554714
Electronic books.

Towards Inclusive Low-Resource Speech Technologies : = A Case Study of Educational Systems for African American English-Speaking Children.
LDR:05374ntm a22004217 4500 001 1146483
005 20240812064627.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798381953404
035 $a (MiAaPQ)AAI31139575
035 $a AAI31139575
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Johnson, Alexander. $3 1471876
245 1 0 $a Towards Inclusive Low-Resource Speech Technologies : $b A Case Study of Educational Systems for African American English-Speaking Children.
264 0 $c 2024
300 $a 1 online resource (119 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-09, Section: A.
500 $a Advisor: Alwan, Abeer A.
502 $a Thesis (Ph.D.)--University of California, Los Angeles, 2024.
504 $a Includes bibliographical references
520 $a The potential of speech technology to improve educational outcomes has been a topic of great interest in recent years. For example, automatic speech recognition (ASR) systems could be employed to provide kindergarten-aged children with real-time feedback on their literacy and pronunciation as they practice reading aloud. Within these systems, speaker identification (SID) technology could additionally be used to identify the user's speaker characteristics in order to ensure that they receive age, language, and dialect-appropriate feedback. While these technologies are more established for well-represented groups in STEM (ie. able-bodied, adult, first-language speakers of mainstream dialects), they give much worse performance for underrepresented groups (young children, speakers of non-mainstream dialects, people with speech-related disabilities, etc.). This work focuses on improving speech technology performance for children's speech and African American English (AAE) dialect speech with the goal of creating more equitable outcomes in early education. The contributions of this work span three primary areas: 1) Dialect identification and density scoring, 2) data augmentation for speech recognition, and 3) Natural Language Processing for fair and inclusive automatic speech assessment.First, we create a robust system for dialect identification of African American English for both children and adult's speech. This system aims to take an input utterance from a speaker of either African American English or Mainstream American English and determine which of the two dialects the utterance belongs. The system fuses features from paralinguistics, self-supervised learning representations, automatic speech recognition system outputs, prosodic contours, and other descriptors of the speech signal in order to learn a mapping from the input acoustic information to a dialect classification decision. We further explore this architecture in automatic dialect density estimation, a task we create and develop. In dialect density scoring, we train a system to automatically predict a speaker's frequency of usage of dialect-specific patterns. This information can then be passed to a speech recognition system for more dialect-informed processing.Second, we develop a data augmentation algorithm to improve zero-shot and few-shot speech recognition of low-resource dialects. The algorithm, named LPCAugment, deconstructs an input speech signal into a source and filter representation using linear predictive coding (LPC) analysis. The poles of the filter representation can then be perturbed independently of the source representation in order to model formant shifts that may be seen across accents and dialects. We use this perturbation method to artificially generate speech samples with shifted formant locations to serve as additional training data for a speech recognition system. This speech recognition system is then evaluated on children's speech for child speakers of a Southern California dialect and child speakers of an Atlanta, Georgia, area dialect.Third, we explore automatic analysis and scoring of speech recognition transcripts for educational assessments. Given information about a student's spoken dialect and automatically generated transcripts of their oral response to an assessment prompt, we train a system to automatically grade the quality of the response with respect to a pre-determined criterion. This system uses language modeling and spoken information retrieval to identify key features in the spoken response and holistically decide if the response aligns with the grading criteria. Combined, the steps in this work form a framework for inclusive spoken language understanding technology that can be used to perform provide students with dialect-appropriate language training or language assessment.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Electrical engineering. $3 596380
650 4 $a Educational technology. $3 556755
650 4 $a Linguistics. $3 557829
650 4 $a African American studies. $3 1180117
650 4 $a Communication. $3 556422
653 $a Speech technology
653 $a Automatic speech recognition
653 $a Speaker identification
653 $a Linear predictive coding
653 $a African American English
655 7 $a Electronic books. $2 local $3 554714
690 $a 0544
690 $a 0459
690 $a 0290
690 $a 0710
690 $a 0296
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a University of California, Los Angeles. $b Electrical and Computer Engineering 0333. $3 1413511
773 0 $t Dissertations Abstracts International $g 85-09A.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31139575 $z click for full text (PQDT)