國立虎尾科技大學 |

A Conversational Agent for Dynamic Procedural Interactions.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	A Conversational Agent for Dynamic Procedural Interactions./
Author:	Colon-Hernandez, Pedro.
Description:	1 online resource (422 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-10, Section: A.
Contained By:	Dissertations Abstracts International85-10A.
Subject:	Fine arts. -
Online resource:	click for full text (PQDT)
ISBN:	9798381958775

A Conversational Agent for Dynamic Procedural Interactions.
Colon-Hernandez, Pedro.

A Conversational Agent for Dynamic Procedural Interactions. - 1 online resource (422 pages)

Source: Dissertations Abstracts International, Volume: 85-10, Section: A.

Thesis (Ph.D.)--Massachusetts Institute of Technology, 2023.

Includes bibliographical references

How-To questions (e.g., "How do I cook rice?", "How do I write a check?", or "How do I send pictures to my family from my iPhone?") are some of the most common questions asked of search engines and presumably of conversational agents as well. Answers to How-To questions should generally be in the form of a procedure; step-by-step instructions that users perform in sequence. However, people find reading instructions cognitively demanding and often prefer that another person guide them through a procedure. Prior work in automating procedural guidance either concentrates on how to communicate instructions or how to reason about procedural knowledge to extract states of entities. In this work, we present an end-to-end procedural voice guidance system that automatically generates and presents step-by-step instructions to users through a conversational agent. This system overcomes three significant challenges: generating a contextual knowledge graph of the procedure, ordering necessary information through reasoning on that graph and converting it to procedural steps, and finally constructing a conversational system that delivers the procedure in a way that is easily followed by users. Our approach improves upon the current state-of-the-art in conversational agents, which often hand off the interaction to a web search. We demonstrate that our system can be utilized for end-user guidance, and that a contextual commonsense inference system can be used for procedural knowledge graph generation and ultimately procedural step generation. We also show that reasoning for procedural step generation is essential for the task. Lastly, we show that combining our knowledge driven system, both its steps and contextual commonsense assertions with a large language model (LLM) provides more accurate and reliable procedural guidance in tasks that the LLM may have trouble recalling/or were created after training. This work opens up paths to perform contextual graph-based reasoning for story-based applications and helps inform the design of future conversational agents within the domain of procedural guidance.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798381958775Subjects--Topical Terms:

1112523
Fine arts.
Subjects--Index Terms:

Large language modelIndex Terms--Genre/Form:

554714
Electronic books.

A Conversational Agent for Dynamic Procedural Interactions.
LDR:03415ntm a22003497 4500 001 1146479
005 20240812064626.5
006 m o d
007 cr bn ---uuuuu
008 250605s2023 xx obm 000 0 eng d
020 $a 9798381958775
035 $a (MiAaPQ)AAI31091485
035 $a (MiAaPQ)MIT1721_1_151990
035 $a AAI31091485
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Colon-Hernandez, Pedro. $3 1471872
245 1 2 $a A Conversational Agent for Dynamic Procedural Interactions.
264 0 $c 2023
300 $a 1 online resource (422 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-10, Section: A.
500 $a Advisor: Breazeal, Cynthia.
502 $a Thesis (Ph.D.)--Massachusetts Institute of Technology, 2023.
504 $a Includes bibliographical references
520 $a How-To questions (e.g., "How do I cook rice?", "How do I write a check?", or "How do I send pictures to my family from my iPhone?") are some of the most common questions asked of search engines and presumably of conversational agents as well. Answers to How-To questions should generally be in the form of a procedure; step-by-step instructions that users perform in sequence. However, people find reading instructions cognitively demanding and often prefer that another person guide them through a procedure. Prior work in automating procedural guidance either concentrates on how to communicate instructions or how to reason about procedural knowledge to extract states of entities. In this work, we present an end-to-end procedural voice guidance system that automatically generates and presents step-by-step instructions to users through a conversational agent. This system overcomes three significant challenges: generating a contextual knowledge graph of the procedure, ordering necessary information through reasoning on that graph and converting it to procedural steps, and finally constructing a conversational system that delivers the procedure in a way that is easily followed by users. Our approach improves upon the current state-of-the-art in conversational agents, which often hand off the interaction to a web search. We demonstrate that our system can be utilized for end-user guidance, and that a contextual commonsense inference system can be used for procedural knowledge graph generation and ultimately procedural step generation. We also show that reasoning for procedural step generation is essential for the task. Lastly, we show that combining our knowledge driven system, both its steps and contextual commonsense assertions with a large language model (LLM) provides more accurate and reliable procedural guidance in tasks that the LLM may have trouble recalling/or were created after training. This work opens up paths to perform contextual graph-based reasoning for story-based applications and helps inform the design of future conversational agents within the domain of procedural guidance.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Fine arts. $3 1112523
653 $a Large language model
653 $a Dynamic procedural interactions
655 7 $a Electronic books. $2 local $3 554714
690 $a 0357
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a Massachusetts Institute of Technology. $3 574355
773 0 $t Dissertations Abstracts International $g 85-10A.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31091485 $z click for full text (PQDT)