國立虎尾科技大學 |

Model-Agnostic Trajectory Abstraction and Visualization Method for Explainability in Reinforcement Learning.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Model-Agnostic Trajectory Abstraction and Visualization Method for Explainability in Reinforcement Learning./
Author:	Takagi, Yoshiki.
Description:	1 online resource (105 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-12, Section: B.
Contained By:	Dissertations Abstracts International85-12B.
Subject:	Computer science. -
Online resource:	click for full text (PQDT)
ISBN:	9798383111680

Model-Agnostic Trajectory Abstraction and Visualization Method for Explainability in Reinforcement Learning.
Takagi, Yoshiki.

Model-Agnostic Trajectory Abstraction and Visualization Method for Explainability in Reinforcement Learning. - 1 online resource (105 pages)

Source: Dissertations Abstracts International, Volume: 85-12, Section: B.

Thesis (Ph.D.)--University of Hawai'i at Manoa, 2024.

Includes bibliographical references

Reinforcement learning (RL) has evolved rapidly in the past decade and is now capable of achieving human capabilities, such as self-driving cars. Moreover, in the last few years, the performance of deep RL, which applies deep neural networks to RL, has surpassed that of skilled human players in areas of video games, chess, and Go. However, as deep RL models become more complex, understanding and interpreting these models poses significant challenges.Explainable AI (XAI) research has shown the potential to close the gap between humans and a deep RL agent by providing explanations that help users to understand how the agent works. XAI approaches have been tailored for both RL experts and non-experts. For RL experts, visualizations of internal agent parameters reveal the learning mechanisms of deep RL agents, offering precise insights into agent behavior. However, this approach is less accessible to users who do not have RL expertise (non-RL experts). The communication gap between RL experts and non-experts thus remains a critical issue. For example, in discussions about the decision boundaries of autonomous Unmanned Aerial Vehicles (UAVs) between RL practitioners and pilots, the following issues arise:Pilots, who are non-RL experts, have domain knowledge, but they cannot use XAI interfaces designed for RL experts in the assessment of the RL model; In order to obtain feedback from pilots, RL experts need to explain the behavior of the RL model while minimizing the use of RL terminology;Pilots may use domain specific terminology during the assessment and the RL expert needs to interpret the pilot's statements and apply them to the model;Therefore, the central questions are: How can both RL experts and non-RL experts understand the behavior of an agent? In other words, how can humans naturally build a mental model of an agent? A promising approach is the 'familiarization effect' from cognitive psychology, where exposure to an agent's behavior in various scenarios helps users intuitively understand the agent, which is later applied to Human Robot Interaction. For instance, one research group observed that watching a robot's trajectory in videos enables users to predict the robot's future trajectory. Another study pointed out that short video clips of an agents' game-play can effectively build mental models of the agents' performance. However, this strategy may be less effective with multiple agents or in complex, extended tasks due to human limitations in short-term visual memory. Therefore, this dissertation addresses this problem by proposing a trajectory visualization that gives a high-level view of agents' behaviors through an abstraction of agents' behavior. This research will open up new directions, such as that domain experts who are not familiar with RL can get more involved in the development of RL which can lead to identifying important agent's behavior patterns that cannot be recognized by RL experts alone, and that the possibility of allowing general users to assess the capabilities and limitations of agents in the task of monitoring self-driving agents as a driver.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798383111680Subjects--Topical Terms:

573171
Computer science.
Subjects--Index Terms:

Explainable AIIndex Terms--Genre/Form:

554714
Electronic books.

Model-Agnostic Trajectory Abstraction and Visualization Method for Explainability in Reinforcement Learning.
LDR:04590ntm a22004097 4500 001 1150138
005 20241022111556.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798383111680
035 $a (MiAaPQ)AAI30993031
035 $a AAI30993031
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Takagi, Yoshiki. $3 1476570
245 1 0 $a Model-Agnostic Trajectory Abstraction and Visualization Method for Explainability in Reinforcement Learning.
264 0 $c 2024
300 $a 1 online resource (105 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-12, Section: B.
500 $a Advisor: Leigh, Jason.
502 $a Thesis (Ph.D.)--University of Hawai'i at Manoa, 2024.
504 $a Includes bibliographical references
520 $a Reinforcement learning (RL) has evolved rapidly in the past decade and is now capable of achieving human capabilities, such as self-driving cars. Moreover, in the last few years, the performance of deep RL, which applies deep neural networks to RL, has surpassed that of skilled human players in areas of video games, chess, and Go. However, as deep RL models become more complex, understanding and interpreting these models poses significant challenges.Explainable AI (XAI) research has shown the potential to close the gap between humans and a deep RL agent by providing explanations that help users to understand how the agent works. XAI approaches have been tailored for both RL experts and non-experts. For RL experts, visualizations of internal agent parameters reveal the learning mechanisms of deep RL agents, offering precise insights into agent behavior. However, this approach is less accessible to users who do not have RL expertise (non-RL experts). The communication gap between RL experts and non-experts thus remains a critical issue. For example, in discussions about the decision boundaries of autonomous Unmanned Aerial Vehicles (UAVs) between RL practitioners and pilots, the following issues arise:Pilots, who are non-RL experts, have domain knowledge, but they cannot use XAI interfaces designed for RL experts in the assessment of the RL model; In order to obtain feedback from pilots, RL experts need to explain the behavior of the RL model while minimizing the use of RL terminology;Pilots may use domain specific terminology during the assessment and the RL expert needs to interpret the pilot's statements and apply them to the model;Therefore, the central questions are: How can both RL experts and non-RL experts understand the behavior of an agent? In other words, how can humans naturally build a mental model of an agent? A promising approach is the 'familiarization effect' from cognitive psychology, where exposure to an agent's behavior in various scenarios helps users intuitively understand the agent, which is later applied to Human Robot Interaction. For instance, one research group observed that watching a robot's trajectory in videos enables users to predict the robot's future trajectory. Another study pointed out that short video clips of an agents' game-play can effectively build mental models of the agents' performance. However, this strategy may be less effective with multiple agents or in complex, extended tasks due to human limitations in short-term visual memory. Therefore, this dissertation addresses this problem by proposing a trajectory visualization that gives a high-level view of agents' behaviors through an abstraction of agents' behavior. This research will open up new directions, such as that domain experts who are not familiar with RL can get more involved in the development of RL which can lead to identifying important agent's behavior patterns that cannot be recognized by RL experts alone, and that the possibility of allowing general users to assess the capabilities and limitations of agents in the task of monitoring self-driving agents as a driver.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Computer science. $3 573171
650 4 $a Computer engineering. $3 569006
650 4 $a Automotive engineering. $3 1104081
653 $a Explainable AI
653 $a Human-computer interaction
653 $a Reinforcement learning
653 $a Storyline visualization
653 $a Trajectory abstraction
655 7 $a Electronic books. $2 local $3 554714
690 $a 0984
690 $a 0464
690 $a 0800
690 $a 0540
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a University of Hawai'i at Manoa. $b Computer Science. $3 1476571
773 0 $t Dissertations Abstracts International $g 85-12B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30993031 $z click for full text (PQDT)