國立虎尾科技大學 |

High-Performance Learning Systems Using Low-Precision Nanoscale Devices.

Record Type:	Language materials, printed : Monograph/item
Title/Author:	High-Performance Learning Systems Using Low-Precision Nanoscale Devices./
Author:	Sasidharan Rajalekshmi, Nandakumar.
Published:	Ann Arbor : ProQuest Dissertations & Theses, : 2019,
Description:	188 p.
Notes:	Source: Dissertations Abstracts International, Volume: 81-02, Section: B.
Contained By:	Dissertations Abstracts International81-02B.
Subject:	Electrical engineering. -
Online resource:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=13418861
ISBN:	9781085562461

High-Performance Learning Systems Using Low-Precision Nanoscale Devices.
Sasidharan Rajalekshmi, Nandakumar.

High-Performance Learning Systems Using Low-Precision Nanoscale Devices. - Ann Arbor : ProQuest Dissertations & Theses, 2019 - 188 p.

Source: Dissertations Abstracts International, Volume: 81-02, Section: B.

Thesis (Ph.D.)--New Jersey Institute of Technology, 2019.

This item must not be sold to any third party vendors.

Brain-inspired computation promises a paradigm shift in information processing, both in terms of its parallel processing architecture and the ability to learn to tackle problems deemed unsolvable by traditional algorithmic approaches. The computational capability of the human brain is believed to stem from an interconnected network of 100 billion compute nodes (neurons) that interact with each other through approximately 1015 adjustable memory junctions (synapses). The conductance of synapses is modifiable allowing the network to learn and perform various cognitive functions. Artificial neural networks inspired by this architecture have demonstrated even super-human performance in many complex tasks.Computational systems based on the von Neumann architecture, however, are ill-suited to optimize and operate these large networks, as they have to constantly move data between the physically separated processor and memory units. Crossbar arrays of nanoscale analog memory devices could store large network weight matrices in their respective conductances and could perform matrix operations without moving the weights to a processor. While this `in-memory computation' provides an efficient and scalable architecture, the trainability of the memory devices is constrained by their limited precision, stochasticity, and non-linearity, and therefore poses a major challenge.In this thesis, a mixed-precision architecture is demonstrated which uses a high-precision digital memory to compensate for the limited precision of the synaptic devices during the training of deep neural networks. In the proposed architecture, the desired weight updates are accumulated in high-precision and transferred to the synaptic devices when the accumulated update exceeds a threshold representing the average device update granularity. Deep neural networks based on experimental nanoscale devices are shown to achieve performance comparable to high-precision software simulations by this approach.Phase-change memory devices (PCM) on a prototype chip from IBM is used to experimentally demonstrate the proposed architecture. Artificial neural networks whose synapses are realized using PCM devices are trained to classify handwritten images from the MNIST dataset and the mixed-precision approach is successful in achieving training accuracies comparable to floating-point simulations. On-chip inference experiment using the PCM devices shows that the network states are retained reliably for more than 106s. The architecture is estimated to achieve approximately 20 times acceleration in training these networks compared to high-precision implementations and has a potential for at least 100 times efficiency gain in inference.Supervised training and inference of third generation spiking neural networks using PCM are also demonstrated using the hardware platform. New array level conductance scaling methods are demonstrated for adaptive mapping of the device conductance to network weights and to compensate for the effect of conductance drift. During the course of the study, Ge2Sb2Te5 based PCM and Cu/SiO2/W based resistive random access memories are characterized for their gradual conductance modulation behavior and statistically accurate models are created. The models are used to pre-validate the experiments and to test the efficacy of different synapse configurations in the training of neural networks.Collectively, this work demonstrates the feasibility of realizing high-performance learning systems that use low-precision nanoscale memory devices, with accuracies comparable to those obtained from high-precision software training. Such learning systems could have widespread applications including for energy and memory constrained edge computing and internet of things.

ISBN: 9781085562461Subjects--Topical Terms:

596380
Electrical engineering.
Subjects--Index Terms:

Deep learning

High-Performance Learning Systems Using Low-Precision Nanoscale Devices.
LDR:05012nam a2200397 4500 001 951769
005 20200821052152.5
008 200914s2019 ||||||||||||||||| ||eng d
020 $a 9781085562461
035 $a (MiAaPQ)AAI13418861
035 $a AAI13418861
040 $a MiAaPQ $c MiAaPQ
100 1 $a Sasidharan Rajalekshmi, Nandakumar. $3 1241233
245 1 0 $a High-Performance Learning Systems Using Low-Precision Nanoscale Devices.
260 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2019
300 $a 188 p.
500 $a Source: Dissertations Abstracts International, Volume: 81-02, Section: B.
500 $a Advisor: Rajendran, Bipin.
502 $a Thesis (Ph.D.)--New Jersey Institute of Technology, 2019.
506 $a This item must not be sold to any third party vendors.
520 $a Brain-inspired computation promises a paradigm shift in information processing, both in terms of its parallel processing architecture and the ability to learn to tackle problems deemed unsolvable by traditional algorithmic approaches. The computational capability of the human brain is believed to stem from an interconnected network of 100 billion compute nodes (neurons) that interact with each other through approximately 1015 adjustable memory junctions (synapses). The conductance of synapses is modifiable allowing the network to learn and perform various cognitive functions. Artificial neural networks inspired by this architecture have demonstrated even super-human performance in many complex tasks.Computational systems based on the von Neumann architecture, however, are ill-suited to optimize and operate these large networks, as they have to constantly move data between the physically separated processor and memory units. Crossbar arrays of nanoscale analog memory devices could store large network weight matrices in their respective conductances and could perform matrix operations without moving the weights to a processor. While this `in-memory computation' provides an efficient and scalable architecture, the trainability of the memory devices is constrained by their limited precision, stochasticity, and non-linearity, and therefore poses a major challenge.In this thesis, a mixed-precision architecture is demonstrated which uses a high-precision digital memory to compensate for the limited precision of the synaptic devices during the training of deep neural networks. In the proposed architecture, the desired weight updates are accumulated in high-precision and transferred to the synaptic devices when the accumulated update exceeds a threshold representing the average device update granularity. Deep neural networks based on experimental nanoscale devices are shown to achieve performance comparable to high-precision software simulations by this approach.Phase-change memory devices (PCM) on a prototype chip from IBM is used to experimentally demonstrate the proposed architecture. Artificial neural networks whose synapses are realized using PCM devices are trained to classify handwritten images from the MNIST dataset and the mixed-precision approach is successful in achieving training accuracies comparable to floating-point simulations. On-chip inference experiment using the PCM devices shows that the network states are retained reliably for more than 106s. The architecture is estimated to achieve approximately 20 times acceleration in training these networks compared to high-precision implementations and has a potential for at least 100 times efficiency gain in inference.Supervised training and inference of third generation spiking neural networks using PCM are also demonstrated using the hardware platform. New array level conductance scaling methods are demonstrated for adaptive mapping of the device conductance to network weights and to compensate for the effect of conductance drift. During the course of the study, Ge2Sb2Te5 based PCM and Cu/SiO2/W based resistive random access memories are characterized for their gradual conductance modulation behavior and statistically accurate models are created. The models are used to pre-validate the experiments and to test the efficacy of different synapse configurations in the training of neural networks.Collectively, this work demonstrates the feasibility of realizing high-performance learning systems that use low-precision nanoscale memory devices, with accuracies comparable to those obtained from high-precision software training. Such learning systems could have widespread applications including for energy and memory constrained edge computing and internet of things.
590 $a School code: 0152.
650 4 $a Electrical engineering. $3 596380
650 4 $a Neurosciences. $3 593561
650 4 $a Computer engineering. $3 569006
650 4 $a Nanotechnology. $3 557660
653 $a Deep learning
653 $a Memristor
653 $a Mixed-precision
653 $a Neuromorphic engineering
653 $a Phase change memory
653 $a Spiking neural network
690 $a 0544
690 $a 0317
690 $a 0464
690 $a 0652
710 2 $a New Jersey Institute of Technology. $b Helen and John C. Hartmann Department of Electrical and Computer Engineering. $3 1241234
773 0 $t Dissertations Abstracts International $g 81-02B.
790 $a 0152
791 $a Ph.D.
792 $a 2019
793 $a English
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=13418861