國立虎尾科技大學 |

Deep Reinforcement Learning with Consensus for Manipulators.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Deep Reinforcement Learning with Consensus for Manipulators./
Author:	Liu, Wenxing.
Description:	1 online resource (212 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-09, Section: B.
Contained By:	Dissertations Abstracts International85-09B.
Subject:	Kinematics. -
Online resource:	click for full text (PQDT)
ISBN:	9798381840339

Deep Reinforcement Learning with Consensus for Manipulators.
Liu, Wenxing.

Deep Reinforcement Learning with Consensus for Manipulators. - 1 online resource (212 pages)

Source: Dissertations Abstracts International, Volume: 85-09, Section: B.

Thesis (Ph.D.)--The University of Manchester (United Kingdom), 2023.

Includes bibliographical references

With the development of industrialization, the working environment of robotics gradually becomes complex, diverse, and fast. Most manipulators at present are still designed for simple action repetition, which means that the working environment is determined and the target should be relatively fixed. Therefore, they lack the ability to perceive the surrounding environment. The main purpose of this thesis is to develop consensus-based training and deep reinforcement learning methods that enable robot arms to interact with the environment autonomously.First of all, a model-free off-policy actor-critic based deep reinforcement learning method is proposed to solve the classical path planning problem of a UR5 robot arm. The proposed method not only guarantees that the joint angle of the UR5 robotic arm lies within the allowable range each time when it reaches the random target point, but also ensures that the joint angle of the UR5 robotic arm is always within the allowable range during the entire episode of training.Moreover, a self-supervised vision-based deep reinforcement learning method that allows robots to pick and place objects effectively and efficiently when directly transferring a training model from simulation to the real world is demonstrated. A heightsensitive action policy is specially designed for the proposed method to deal with crowded and stacked objects in challenging environments. The training model with the proposed approach can be applied directly to a real suction task without any finetuning from the real world while maintaining a high suction success rate. It is also validated that the training model can be deployed to suction novel objects in a real experiment with a suction success rate of 90% without any real-world fine-tuning.Additionally, an algorithm that combines actor-critic based off-policy method with consensus-based distributed training is proposed to deal with multi-agent deep reinforcement learning problems. Specially, a convergence analysis of a consensus algorithm for a type of nonlinear systems with a Lyapunov method is developed, and this result is used to analyse the convergence properties of the actor and the critic training parameters. To validate the implementation of the proposed algorithm, a multi-agent training framework is proposed to train each UR5 robot arm to reach the random target position. Experiments are provided to demonstrate the effectiveness and feasibility of the proposed algorithm.Finally, a Consensus-based Sim-and-Real deep reinforcement learning algorithm is developed for manipulator pick-and-place tasks. Agents are trained in both simulators and the real environment simultaneously to get the optimal policies for both sim-and-real worlds. The proposed algorithm saves required training time and shows comparable performance in both sim-and-real worlds.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798381840339Subjects--Topical Terms:

792042
Kinematics.
Index Terms--Genre/Form:

554714
Electronic books.

Deep Reinforcement Learning with Consensus for Manipulators.
LDR:04211ntm a22003737 4500 001 1149681
005 20241022112613.5
006 m o d
007 cr bn ---uuuuu
008 250605s2023 xx obm 000 0 eng d
020 $a 9798381840339
035 $a (MiAaPQ)AAI30841290
035 $a (MiAaPQ)Manchester_UK66d7b7bc-6283-4385-b1b0-520dc46ea43d
035 $a AAI30841290
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Liu, Wenxing. $3 1476007
245 1 0 $a Deep Reinforcement Learning with Consensus for Manipulators.
264 0 $c 2023
300 $a 1 online resource (212 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-09, Section: B.
500 $a Advisor: Carrasco , Joaquin;Herrmann, Guido.
502 $a Thesis (Ph.D.)--The University of Manchester (United Kingdom), 2023.
504 $a Includes bibliographical references
520 $a With the development of industrialization, the working environment of robotics gradually becomes complex, diverse, and fast. Most manipulators at present are still designed for simple action repetition, which means that the working environment is determined and the target should be relatively fixed. Therefore, they lack the ability to perceive the surrounding environment. The main purpose of this thesis is to develop consensus-based training and deep reinforcement learning methods that enable robot arms to interact with the environment autonomously.First of all, a model-free off-policy actor-critic based deep reinforcement learning method is proposed to solve the classical path planning problem of a UR5 robot arm. The proposed method not only guarantees that the joint angle of the UR5 robotic arm lies within the allowable range each time when it reaches the random target point, but also ensures that the joint angle of the UR5 robotic arm is always within the allowable range during the entire episode of training.Moreover, a self-supervised vision-based deep reinforcement learning method that allows robots to pick and place objects effectively and efficiently when directly transferring a training model from simulation to the real world is demonstrated. A heightsensitive action policy is specially designed for the proposed method to deal with crowded and stacked objects in challenging environments. The training model with the proposed approach can be applied directly to a real suction task without any finetuning from the real world while maintaining a high suction success rate. It is also validated that the training model can be deployed to suction novel objects in a real experiment with a suction success rate of 90% without any real-world fine-tuning.Additionally, an algorithm that combines actor-critic based off-policy method with consensus-based distributed training is proposed to deal with multi-agent deep reinforcement learning problems. Specially, a convergence analysis of a consensus algorithm for a type of nonlinear systems with a Lyapunov method is developed, and this result is used to analyse the convergence properties of the actor and the critic training parameters. To validate the implementation of the proposed algorithm, a multi-agent training framework is proposed to train each UR5 robot arm to reach the random target position. Experiments are provided to demonstrate the effectiveness and feasibility of the proposed algorithm.Finally, a Consensus-based Sim-and-Real deep reinforcement learning algorithm is developed for manipulator pick-and-place tasks. Agents are trained in both simulators and the real environment simultaneously to get the optimal policies for both sim-and-real worlds. The proposed algorithm saves required training time and shows comparable performance in both sim-and-real worlds.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Kinematics. $3 792042
650 4 $a Deep learning. $3 1127428
650 4 $a Teaching methods. $3 1466786
650 4 $a Success. $3 571577
650 4 $a Computer aided design--CAD. $3 1372801
650 4 $a Neural networks. $3 1011215
650 4 $a Robots. $3 654842
650 4 $a Eigenvalues. $3 527710
650 4 $a Feedback. $3 1047630
650 4 $a Robotics. $3 561941
650 4 $a Design. $3 595500
650 4 $a Pedagogy. $3 1148703
650 4 $a Education. $3 555912
655 7 $a Electronic books. $2 local $3 554714
690 $a 0771
690 $a 0800
690 $a 0389
690 $a 0456
690 $a 0515
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a The University of Manchester (United Kingdom). $3 1213791
773 0 $t Dissertations Abstracts International $g 85-09B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30841290 $z click for full text (PQDT)