國立虎尾科技大學 |

Automated Parallelization to Improve Usability and Efficiency of Distributed Neural Network Training.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Automated Parallelization to Improve Usability and Efficiency of Distributed Neural Network Training./
Author:	Grabaskas, Nathaniel J.
Description:	1 online resource (108 pages)
Notes:	Source: Masters Abstracts International, Volume: 57-05.
Contained By:	Masters Abstracts International57-05(E).
Subject:	Computer science. -
Online resource:	click for full text (PQDT)
ISBN:	9780355850390

Automated Parallelization to Improve Usability and Efficiency of Distributed Neural Network Training.
Grabaskas, Nathaniel J.

Automated Parallelization to Improve Usability and Efficiency of Distributed Neural Network Training. - 1 online resource (108 pages)

Source: Masters Abstracts International, Volume: 57-05.

Thesis (Master's)--University of Washington, 2018.

Includes bibliographical references

The recent success of Deep Neural Networks (DNNs) has triggered a race to build larger and larger DNNs; however, a known limitation is the training speed. To solve this speed problem, distributed neural network training has become an increasingly large area of research. Usability, the complexity for a machine learning or data scientist to implement distributed neural network training, is an aspect rarely considered, yet critical. There is strong evidence growing complexity has a direct impact on development effort, maintainability, and fault proneness of software. We investigated, if automation can greatly reduce the implementation complexity of distributing neural network training across multiple devices without loss of computational efficiency when compared to manual parallelization. Experiments were conducted using Convolutional Neural Networks (CNN) and Multi-Layer Perceptron (MLP) networks to perform image classification on CIFAR-10 and MNIST datasets. Hardware consisted of an embedded, four node NVIDIA Jetson TX1 cluster. Torch Automatic Distributed Neural Network (TorchAD-NN) reduces the implementation complexity of data parallel neural network training by more than 90% and providing components, with near zero implementation complexity, to easily parallelize all or only select fully-connected neural layers.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2018

Mode of access: World Wide Web

ISBN: 9780355850390Subjects--Topical Terms:

573171
Computer science.
Index Terms--Genre/Form:

554714
Electronic books.

Automated Parallelization to Improve Usability and Efficiency of Distributed Neural Network Training.
LDR:02573ntm a2200337Ki 4500 001 916869
005 20180928111502.5
006 m o u
007 cr mn||||a|a||
008 190606s2018 xx obm 000 0 eng d
020 $a 9780355850390
035 $a (MiAaPQ)AAI10750530
035 $a (MiAaPQ)washington:18319
035 $a AAI10750530
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Grabaskas, Nathaniel J. $3 1190726
245 1 0 $a Automated Parallelization to Improve Usability and Efficiency of Distributed Neural Network Training.
264 0 $c 2018
300 $a 1 online resource (108 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Masters Abstracts International, Volume: 57-05.
500 $a Adviser: Munehiro Fukuda.
502 $a Thesis (Master's)--University of Washington, 2018.
504 $a Includes bibliographical references
520 $a The recent success of Deep Neural Networks (DNNs) has triggered a race to build larger and larger DNNs; however, a known limitation is the training speed. To solve this speed problem, distributed neural network training has become an increasingly large area of research. Usability, the complexity for a machine learning or data scientist to implement distributed neural network training, is an aspect rarely considered, yet critical. There is strong evidence growing complexity has a direct impact on development effort, maintainability, and fault proneness of software. We investigated, if automation can greatly reduce the implementation complexity of distributing neural network training across multiple devices without loss of computational efficiency when compared to manual parallelization. Experiments were conducted using Convolutional Neural Networks (CNN) and Multi-Layer Perceptron (MLP) networks to perform image classification on CIFAR-10 and MNIST datasets. Hardware consisted of an embedded, four node NVIDIA Jetson TX1 cluster. Torch Automatic Distributed Neural Network (TorchAD-NN) reduces the implementation complexity of data parallel neural network training by more than 90% and providing components, with near zero implementation complexity, to easily parallelize all or only select fully-connected neural layers.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2018
538 $a Mode of access: World Wide Web
650 4 $a Computer science. $3 573171
650 4 $a Artificial intelligence. $3 559380
655 7 $a Electronic books. $2 local $3 554714
690 $a 0984
690 $a 0800
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a University of Washington. $b Computing and Software System. $3 1179443
773 0 $t Masters Abstracts International $g 57-05(E).
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=10750530 $z click for full text (PQDT)