國立虎尾科技大學 |

Towards Efficient Deep Neural Network Inference and Training for Ubiquitous AI.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Towards Efficient Deep Neural Network Inference and Training for Ubiquitous AI./
Author:	Yuan, Geng.
Description:	1 online resource (124 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-02, Section: B.
Contained By:	Dissertations Abstracts International85-02B.
Subject:	Computer engineering. -
Online resource:	click for full text (PQDT)
ISBN:	9798380156622

Towards Efficient Deep Neural Network Inference and Training for Ubiquitous AI.
Yuan, Geng.

Towards Efficient Deep Neural Network Inference and Training for Ubiquitous AI. - 1 online resource (124 pages)

Source: Dissertations Abstracts International, Volume: 85-02, Section: B.

Thesis (Ph.D.)--Northeastern University, 2023.

Includes bibliographical references

Machine learning has become increasingly popular in recent years. Due to the high accuracy and excellent scalability, deep neural networks have emerged as a fundamental element in machine learning-based applications, such as autonomous driving, augmented reality, natural language processing, etc. And to promote portability and flexibility, more applications are now targeted on resource-limited edge devices, such as smartphones or IoT devices. However, deploying DNN models on resource-limited devices is very challenging. A deep and large model usually requires a lot of computation resources and huge power consumption as well. As a result, the huge model size, intensive computation, and high power consumption will be the biggest challenges for the genuine democratization of AI. To solve this problem, my research during my Ph.D. study mainly focuses on optimizing and improving AI systems from the perspective of model size, training and inference computation cost, execution latency, and energy efficiency. And my work includes algorithm optimization, architecture design, and the incorporation of emerging devices.In the first part of this dissertation, I will present an algorithm-architecture co-design for the hardware-based DNN accelerator using emerging technology for efficient DNN inference. Recent work demonstrated the promise of using resistive random access memory (ReRAM) as an emerging technology to perform inherently parallel analog domain in-situ matrix-vector multiplication. One key problem is the weights that are signed values. However, in a ReRAM crossbar, weights are stored as conductance of the crossbar cells, and the in-situ computation assumes all cells on each crossbar column are of the same sign. The current architectures either use two ReRAM crossbars for positive and negative weights, or add an offset to weights so that all values become positive. Neither solution is ideal: they either double the cost of crossbars, or incur extra offset circuity. To better address this problem, we propose FORMS, a fine-grained ReRAM-based DNN accelerator with algorithm/hardware co-design. Instead of trying to represent the positive/negative weights, our key design principle is to enforce exactly what is assumed in the in-situ computation-ensuring that all weights in the same column of a crossbar have the same sign. It naturally avoids the cost of an additional crossbar. Besides, we also design a zero-skipping logic, which is especially suitable for the fine-grained computation paradigm, and can effectively skip unnecessary computations, thereby accelerating the inference speed of DNNs.In the second part, we investigate the efficient training technique to enhance the adaptiveness of DNN applications. Recently, sparse training has emerged as a promising paradigm for efficient deep learning on edge devices. The current research mainly devotes efforts to reducing training costs by further increasing model sparsity. However, increasing sparsity is not always ideal since it will inevitably introduce severe accuracy degradation at an extremely high sparsity level. To mitigate this issue, we intend to explore other possible directions to effectively and efficiently reduce sparse training costs while preserving accuracy. To this end, we investigate two techniques, namely, layer freezing and data sieving. First, the layer freezing approach has shown its success in dense model training and fine-tuning, yet it has never been adopted in the sparse training domain. Nevertheless, the unique characteristics of sparse training may hinder the incorporation of layer-freezing techniques. Therefore, we analyze the feasibility and potentiality of using the layer freezing technique in sparse training and find it has the potential to save considerable training costs. Second, we propose a data-sieving method for dataset-efficient training, which further reduces training costs by ensuring only a partial dataset is used throughout the entire training process. We show that both techniques can be well incorporated into the sparse training algorithm to form a generic framework, which we dub SpFDE. Our extensive experiments demonstrate that SpFDE can significantly reduce training costs while preserving accuracy from three dimensions: weight sparsity, layer freezing, and dataset sieving.Model compression strategies, including weight quantization and pruning, are widely recognized as effective approaches to significantly reduce computation and memory intensities, and have been implemented in many DNNs on edge devices. However, most state-of-the-art works focus on ad-hoc optimizations, and there lacks a thorough study to comprehensively reveal the potentials and constraints of different edge devices when considering different compression strategies. In the third part of this dissertation, we qualitatively and quantitatively compare the energy efficiency of FPGA-based and mobile-based DNN executions using mobile GPU and provide a detailed analysis. Based on the observations obtained from the analysis, we propose a unified optimization framework using block-based pruning to reduce the weight storage and accelerate the inference speed on mobile devices and FPGAs, achieving high hardware performance and energy-efficiency gain while maintaining accuracy.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798380156622Subjects--Topical Terms:

569006
Computer engineering.
Subjects--Index Terms:

Deep learningIndex Terms--Genre/Form:

554714
Electronic books.

Towards Efficient Deep Neural Network Inference and Training for Ubiquitous AI.
LDR:06595ntm a22003617 4500 001 1148719
005 20240930100126.5
006 m o d
007 cr bn ---uuuuu
008 250605s2023 xx obm 000 0 eng d
020 $a 9798380156622
035 $a (MiAaPQ)AAI30637002
035 $a AAI30637002
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Yuan, Geng. $3 1474759
245 1 0 $a Towards Efficient Deep Neural Network Inference and Training for Ubiquitous AI.
264 0 $c 2023
300 $a 1 online resource (124 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-02, Section: B.
500 $a Advisor: Wang, Yanzhi.
502 $a Thesis (Ph.D.)--Northeastern University, 2023.
504 $a Includes bibliographical references
520 $a Machine learning has become increasingly popular in recent years. Due to the high accuracy and excellent scalability, deep neural networks have emerged as a fundamental element in machine learning-based applications, such as autonomous driving, augmented reality, natural language processing, etc. And to promote portability and flexibility, more applications are now targeted on resource-limited edge devices, such as smartphones or IoT devices. However, deploying DNN models on resource-limited devices is very challenging. A deep and large model usually requires a lot of computation resources and huge power consumption as well. As a result, the huge model size, intensive computation, and high power consumption will be the biggest challenges for the genuine democratization of AI. To solve this problem, my research during my Ph.D. study mainly focuses on optimizing and improving AI systems from the perspective of model size, training and inference computation cost, execution latency, and energy efficiency. And my work includes algorithm optimization, architecture design, and the incorporation of emerging devices.In the first part of this dissertation, I will present an algorithm-architecture co-design for the hardware-based DNN accelerator using emerging technology for efficient DNN inference. Recent work demonstrated the promise of using resistive random access memory (ReRAM) as an emerging technology to perform inherently parallel analog domain in-situ matrix-vector multiplication. One key problem is the weights that are signed values. However, in a ReRAM crossbar, weights are stored as conductance of the crossbar cells, and the in-situ computation assumes all cells on each crossbar column are of the same sign. The current architectures either use two ReRAM crossbars for positive and negative weights, or add an offset to weights so that all values become positive. Neither solution is ideal: they either double the cost of crossbars, or incur extra offset circuity. To better address this problem, we propose FORMS, a fine-grained ReRAM-based DNN accelerator with algorithm/hardware co-design. Instead of trying to represent the positive/negative weights, our key design principle is to enforce exactly what is assumed in the in-situ computation-ensuring that all weights in the same column of a crossbar have the same sign. It naturally avoids the cost of an additional crossbar. Besides, we also design a zero-skipping logic, which is especially suitable for the fine-grained computation paradigm, and can effectively skip unnecessary computations, thereby accelerating the inference speed of DNNs.In the second part, we investigate the efficient training technique to enhance the adaptiveness of DNN applications. Recently, sparse training has emerged as a promising paradigm for efficient deep learning on edge devices. The current research mainly devotes efforts to reducing training costs by further increasing model sparsity. However, increasing sparsity is not always ideal since it will inevitably introduce severe accuracy degradation at an extremely high sparsity level. To mitigate this issue, we intend to explore other possible directions to effectively and efficiently reduce sparse training costs while preserving accuracy. To this end, we investigate two techniques, namely, layer freezing and data sieving. First, the layer freezing approach has shown its success in dense model training and fine-tuning, yet it has never been adopted in the sparse training domain. Nevertheless, the unique characteristics of sparse training may hinder the incorporation of layer-freezing techniques. Therefore, we analyze the feasibility and potentiality of using the layer freezing technique in sparse training and find it has the potential to save considerable training costs. Second, we propose a data-sieving method for dataset-efficient training, which further reduces training costs by ensuring only a partial dataset is used throughout the entire training process. We show that both techniques can be well incorporated into the sparse training algorithm to form a generic framework, which we dub SpFDE. Our extensive experiments demonstrate that SpFDE can significantly reduce training costs while preserving accuracy from three dimensions: weight sparsity, layer freezing, and dataset sieving.Model compression strategies, including weight quantization and pruning, are widely recognized as effective approaches to significantly reduce computation and memory intensities, and have been implemented in many DNNs on edge devices. However, most state-of-the-art works focus on ad-hoc optimizations, and there lacks a thorough study to comprehensively reveal the potentials and constraints of different edge devices when considering different compression strategies. In the third part of this dissertation, we qualitatively and quantitatively compare the energy efficiency of FPGA-based and mobile-based DNN executions using mobile GPU and provide a detailed analysis. Based on the observations obtained from the analysis, we propose a unified optimization framework using block-based pruning to reduce the weight storage and accelerate the inference speed on mobile devices and FPGAs, achieving high hardware performance and energy-efficiency gain while maintaining accuracy.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Computer engineering. $3 569006
653 $a Deep learning
653 $a Edge devices
653 $a Model compression
655 7 $a Electronic books. $2 local $3 554714
690 $a 0464
690 $a 0800
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a Northeastern University. $b Electrical and Computer Engineering. $3 1182193
773 0 $t Dissertations Abstracts International $g 85-02B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30637002 $z click for full text (PQDT)