國立虎尾科技大學 |

Internationalization of Task-Oriented Dialogue Systems.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Internationalization of Task-Oriented Dialogue Systems./
Author:	Moradshahi, Mehrad.
Description:	1 online resource (115 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-06, Section: B.
Contained By:	Dissertations Abstracts International85-06B.
Subject:	Multilingualism. -
Online resource:	click for full text (PQDT)
ISBN:	9798381019070

Internationalization of Task-Oriented Dialogue Systems.
Moradshahi, Mehrad.

Internationalization of Task-Oriented Dialogue Systems. - 1 online resource (115 pages)

Source: Dissertations Abstracts International, Volume: 85-06, Section: B.

Thesis (Ph.D.)--Stanford University, 2023.

Includes bibliographical references

Virtual assistants and Task-oriented Dialogue (ToD) agents are increasingly prevalent due to their utility in daily tasks. Despite the linguistic diversity worldwide, only a few dominant languages are supported by these digital assistants. This restriction is due to the high cost and manual effort required to produce large, hand-annotated datasets to train these agents. Existing low-cost approaches that rely on cross-lingual embeddings or naive machine translation sacrifice a lot of accuracy for data efficiency, and largely fail in creating a usable dialogue agent.This thesis introduces a novel solution to automatically create ToD agents in new languages by leveraging dialogue data in the source language and neural machine translation. The approach is based on automatic entity-aware training data translation, a concise dialogue data representation enabling effective zero-shot training, and a scalable and robust approach for creating end-to-end high-quality fewshot, validation, and test data, minimizing the manual effort needed.To address data scarcity, we use neural machine translation to translate the training dataset from the source to the target language. We show that naive application of this approach would not yield good performance as entities in the input can be mistranslated, transliterated, or omitted and no longer match with that in the annotation. We propose a series of techniques to improve the quality of the dataset by (1) leveraging word alignments from the neural translation model's cross-attention weights to preserve entities and (2) applying automatic data filtering based on textual semantic similarity to exclude poor translations. Using this approach, we create multilingual versions of Schema2QA, a single-turn question-answering dataset, in 10 different languages. Agents trained on our automatically translated data improve upon previous state-of-the-art by 30-40% and comes within 5-8% of the original English agent.Translation is inherently noisy and poses a special challenge in the end-to-end dialogue setting where the amount of natural language encoded grows with each turn. The accumulation of errors can prevent a correct parse for the rest of the dialogue. To address this, we introduce a new distilled dialogue data representation which significantly reduces the amount of natural language encoded and decoded by the model. On the BiToD dataset, using our representation, we found a 14% improvement in Dialogue Success Rate (DSR) in the fewshot setting.The lack of a high-quality realistic testbed for multilingual ToD evaluation has impeded accurate measurement of research progress on the topic. Prior work deployed human translators to either translate or post-edit an automatically translated dataset. However, this was done only for one or two subtasks of a dialogue agent, and training an intractable end-to-end agent was not possible. To address this, we initiated a global effort to extend a large-scale multi-domain dataset, RiSAWOZ (initially in Chinese), to several new languages: English, Korean, French, Hindi, and code-mixed English-Hindi. To ensure the best quality and fluency, we used human post-editing only for the fewshot, validation, and test data. The challenges encountered in creating this dataset at scale led us to create a toolset that makes post-editing for a new language much faster and cheaper. Experiments show that few-shot training achieves 63-88% performance of the original full-shot. The remaining gap motivates further research on multilingual ToD.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798381019070Subjects--Topical Terms:

556374
Multilingualism.
Index Terms--Genre/Form:

554714
Electronic books.

Internationalization of Task-Oriented Dialogue Systems.
LDR:04838ntm a22003737 4500 001 1142123
005 20240414211951.5
006 m o d
007 cr mn ---uuuuu
008 250605s2023 xx obm 000 0 eng d
020 $a 9798381019070
035 $a (MiAaPQ)AAI30726801
035 $a (MiAaPQ)STANFORDkg582jk7231
035 $a AAI30726801
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Moradshahi, Mehrad. $3 1466326
245 1 0 $a Internationalization of Task-Oriented Dialogue Systems.
264 0 $c 2023
300 $a 1 online resource (115 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-06, Section: B.
500 $a Advisor: Lam, Monica;Boneh, Dan;Sadigh, Dorsa.
502 $a Thesis (Ph.D.)--Stanford University, 2023.
504 $a Includes bibliographical references
520 $a Virtual assistants and Task-oriented Dialogue (ToD) agents are increasingly prevalent due to their utility in daily tasks. Despite the linguistic diversity worldwide, only a few dominant languages are supported by these digital assistants. This restriction is due to the high cost and manual effort required to produce large, hand-annotated datasets to train these agents. Existing low-cost approaches that rely on cross-lingual embeddings or naive machine translation sacrifice a lot of accuracy for data efficiency, and largely fail in creating a usable dialogue agent.This thesis introduces a novel solution to automatically create ToD agents in new languages by leveraging dialogue data in the source language and neural machine translation. The approach is based on automatic entity-aware training data translation, a concise dialogue data representation enabling effective zero-shot training, and a scalable and robust approach for creating end-to-end high-quality fewshot, validation, and test data, minimizing the manual effort needed.To address data scarcity, we use neural machine translation to translate the training dataset from the source to the target language. We show that naive application of this approach would not yield good performance as entities in the input can be mistranslated, transliterated, or omitted and no longer match with that in the annotation. We propose a series of techniques to improve the quality of the dataset by (1) leveraging word alignments from the neural translation model's cross-attention weights to preserve entities and (2) applying automatic data filtering based on textual semantic similarity to exclude poor translations. Using this approach, we create multilingual versions of Schema2QA, a single-turn question-answering dataset, in 10 different languages. Agents trained on our automatically translated data improve upon previous state-of-the-art by 30-40% and comes within 5-8% of the original English agent.Translation is inherently noisy and poses a special challenge in the end-to-end dialogue setting where the amount of natural language encoded grows with each turn. The accumulation of errors can prevent a correct parse for the rest of the dialogue. To address this, we introduce a new distilled dialogue data representation which significantly reduces the amount of natural language encoded and decoded by the model. On the BiToD dataset, using our representation, we found a 14% improvement in Dialogue Success Rate (DSR) in the fewshot setting.The lack of a high-quality realistic testbed for multilingual ToD evaluation has impeded accurate measurement of research progress on the topic. Prior work deployed human translators to either translate or post-edit an automatically translated dataset. However, this was done only for one or two subtasks of a dialogue agent, and training an intractable end-to-end agent was not possible. To address this, we initiated a global effort to extend a large-scale multi-domain dataset, RiSAWOZ (initially in Chinese), to several new languages: English, Korean, French, Hindi, and code-mixed English-Hindi. To ensure the best quality and fluency, we used human post-editing only for the fewshot, validation, and test data. The challenges encountered in creating this dataset at scale led us to create a toolset that makes post-editing for a new language much faster and cheaper. Experiments show that few-shot training achieves 63-88% performance of the original full-shot. The remaining gap motivates further research on multilingual ToD.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Multilingualism. $3 556374
650 4 $a Error analysis. $3 1466327
650 4 $a Semantics. $3 555362
650 4 $a Bilingual education. $3 1148431
650 4 $a Education. $3 555912
650 4 $a Language. $3 571568
650 4 $a Logic. $3 558909
650 4 $a Mathematics. $3 527692
655 7 $a Electronic books. $2 local $3 554714
690 $a 0282
690 $a 0515
690 $a 0679
690 $a 0395
690 $a 0405
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a Stanford University. $3 1184533
773 0 $t Dissertations Abstracts International $g 85-06B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30726801 $z click for full text (PQDT)