國立虎尾科技大學 |

Automatic Compiler-Based Optimizations for Deep Neural Networks.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Automatic Compiler-Based Optimizations for Deep Neural Networks./
Author:	Zheng, Bojian.
Description:	1 online resource (120 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-10, Section: A.
Contained By:	Dissertations Abstracts International85-10A.
Subject:	Computer science. -
Online resource:	click for full text (PQDT)
ISBN:	9798382193489

Automatic Compiler-Based Optimizations for Deep Neural Networks.
Zheng, Bojian.

Automatic Compiler-Based Optimizations for Deep Neural Networks. - 1 online resource (120 pages)

Source: Dissertations Abstracts International, Volume: 85-10, Section: A.

Thesis (Ph.D.)--University of Toronto (Canada), 2024.

Includes bibliographical references

Deep neural networks (DNNs) are the current state-of-the-art machine learning algorithms in various application domains. Due to their importance, it is crucial that we guarantee their efficient executions on hardware platforms such as GPUs. In this thesis, we optimize the runtime performance and the device memory consumption of DNNs running on modern GPUs. To make our optimizations generic, automatic, and transparent to the frontend machine learning practitioners, we use compiler-based approaches that achieve the optimization objective by carefully analyzing DNNs' graph representations, tensor expressions, and the hardware platforms on which they run. Compared with manual implementations that require a significant amount of engineering effort, our thesis work, which is made up of three key contributions, requires minimal changes to frontend applications' source code and can be applied generically to various state-of-the-art DNN models.Our first contribution, Grape, is a new graph compiler on graph-based executions for dynamic DNNs on GPUs. Grape addresses the practicability and efficiency challenges of graph-based executions using three key components: an alias predictor, a metadata compressor, and a predication rewriter. It improves the runtime performance of state-of-the-art text generation and speech recognition workloads by up to 2.99x compared with the machine learning framework baseline, and can optimize workloads that are not practical for prior works on graph-based executions, achieving up to 1.82xspeedup over the baseline.Our second contribution, DietCode, is a new tensor program auto-scheduler that efficiently supports dynamic-shape workloads. DietCode addresses the compilation time challenge of auto-scheduling dynamic-shape tensor programs using a shape-generic search space and a micro-kernel cost model. Not only can DietCode reduce the compilation time by up to 5.88x on the state-of-the-art language modeling workload compared with the existing tensor program auto-scheduler, but it also improves the runtime performance by up to 1.70x.Our third contribution, Echo, is a new compiler-based optimization scheme that reduces GPU memory footprint for training state-of-the-art DNNs. We show that by carefully estimating the recomputation's effect on the memory footprint and the runtime overheads, we can significantly reduce the GPU memory footprint by up to 3.13x with only 1% runtime performance overhead, resulting in up to 1.28x faster convergence to the same validation quality. In addition to the system optimization, in Echo, we also build GPU memory profiling tools that accurately pinpoint the GPU memory consumption bottlenecks of DNNs and are integrated into the state-of-the-art machine learning framework.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798382193489Subjects--Topical Terms:

573171
Computer science.
Subjects--Index Terms:

CompilersIndex Terms--Genre/Form:

554714
Electronic books.

Automatic Compiler-Based Optimizations for Deep Neural Networks.
LDR:04175ntm a22004097 4500 001 1151686
005 20241118085734.5
006 m o d
007 cr mn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798382193489
035 $a (MiAaPQ)AAI30818780
035 $a AAI30818780
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Zheng, Bojian. $3 1478499
245 1 0 $a Automatic Compiler-Based Optimizations for Deep Neural Networks.
264 0 $c 2024
300 $a 1 online resource (120 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-10, Section: A.
500 $a Advisor: Pekhimenko, Gennady.
502 $a Thesis (Ph.D.)--University of Toronto (Canada), 2024.
504 $a Includes bibliographical references
520 $a Deep neural networks (DNNs) are the current state-of-the-art machine learning algorithms in various application domains. Due to their importance, it is crucial that we guarantee their efficient executions on hardware platforms such as GPUs. In this thesis, we optimize the runtime performance and the device memory consumption of DNNs running on modern GPUs. To make our optimizations generic, automatic, and transparent to the frontend machine learning practitioners, we use compiler-based approaches that achieve the optimization objective by carefully analyzing DNNs' graph representations, tensor expressions, and the hardware platforms on which they run. Compared with manual implementations that require a significant amount of engineering effort, our thesis work, which is made up of three key contributions, requires minimal changes to frontend applications' source code and can be applied generically to various state-of-the-art DNN models.Our first contribution, Grape, is a new graph compiler on graph-based executions for dynamic DNNs on GPUs. Grape addresses the practicability and efficiency challenges of graph-based executions using three key components: an alias predictor, a metadata compressor, and a predication rewriter. It improves the runtime performance of state-of-the-art text generation and speech recognition workloads by up to 2.99x compared with the machine learning framework baseline, and can optimize workloads that are not practical for prior works on graph-based executions, achieving up to 1.82xspeedup over the baseline.Our second contribution, DietCode, is a new tensor program auto-scheduler that efficiently supports dynamic-shape workloads. DietCode addresses the compilation time challenge of auto-scheduling dynamic-shape tensor programs using a shape-generic search space and a micro-kernel cost model. Not only can DietCode reduce the compilation time by up to 5.88x on the state-of-the-art language modeling workload compared with the existing tensor program auto-scheduler, but it also improves the runtime performance by up to 1.70x.Our third contribution, Echo, is a new compiler-based optimization scheme that reduces GPU memory footprint for training state-of-the-art DNNs. We show that by carefully estimating the recomputation's effect on the memory footprint and the runtime overheads, we can significantly reduce the GPU memory footprint by up to 3.13x with only 1% runtime performance overhead, resulting in up to 1.28x faster convergence to the same validation quality. In addition to the system optimization, in Echo, we also build GPU memory profiling tools that accurately pinpoint the GPU memory consumption bottlenecks of DNNs and are integrated into the state-of-the-art machine learning framework.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Computer science. $3 573171
650 4 $a Computer engineering. $3 569006
650 4 $a Information science. $3 561178
653 $a Compilers
653 $a Deep neural networks
653 $a Machine learning systems
653 $a Memory optimizations
653 $a Performance optimizations
655 7 $a Electronic books. $2 local $3 554714
690 $a 0984
690 $a 0464
690 $a 0723
690 $a 0800
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a University of Toronto (Canada). $b Computer Science. $3 845521
773 0 $t Dissertations Abstracts International $g 85-10A.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30818780 $z click for full text (PQDT)