國立虎尾科技大學 |

Efficient Models and Learning Strategies for Resource-Constrained Systems.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Efficient Models and Learning Strategies for Resource-Constrained Systems./
Author:	Rabbani, Tahseen.
Description:	1 online resource (327 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-12, Section: B.
Contained By:	Dissertations Abstracts International85-12B.
Subject:	Computer science. -
Online resource:	click for full text (PQDT)
ISBN:	9798383173992

Efficient Models and Learning Strategies for Resource-Constrained Systems.
Rabbani, Tahseen.

Efficient Models and Learning Strategies for Resource-Constrained Systems. - 1 online resource (327 pages)

Source: Dissertations Abstracts International, Volume: 85-12, Section: B.

Thesis (Ph.D.)--University of Maryland, College Park, 2024.

Includes bibliographical references

The last decade has seen sharp improvements in the performance of machine learning (ML) models, but at the cost of vastly increased complexity and size of their underlying architectures. Advances in high-performance computing have enabled researchers to train and deploy models composed of hundreds of billions of parameters. However, harvesting the full utility of large models on smaller clients, such as Internet of Things (IoT) devices, without resorting to external hosting will require a significant reduction of parameters and faster, cheaper inference. In addition to augmenting IoT, efficient models and learning paradigms can reduce energy consumption, encourage technological equity, and are well-suited for deployment in real-world applications which require fast response in low-resource settings.To address these challenges, we introduce multiple, novel strategies for (1) reducing the scale of deep neural networks and (2) faster learning. For the size problem (1), we leverage tools such as tensorization, randomized projections, and locality-sensitive hashing to train on reduced representations of large models without sacrificing performance. For learning efficiency (2), we develop algorithms for cheaper forward passes, accelerated PCA, and asynchronous gradient descent. Several of these methods are tailored for federated learning (FL), a private, distributed learning paradigm where data is decentralized among resource-constrained edge clients. We are exclusively concerned with improving efficiency during training - our techniques do not process pre-trained models or require a device to train over an architecture in its full entirety.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798383173992Subjects--Topical Terms:

573171
Computer science.
Subjects--Index Terms:

CompressionIndex Terms--Genre/Form:

554714
Electronic books.

Efficient Models and Learning Strategies for Resource-Constrained Systems.
LDR:03079ntm a22004097 4500 001 1150166
005 20241022111605.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798383173992
035 $a (MiAaPQ)AAI31144938
035 $a AAI31144938
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Rabbani, Tahseen. $3 1476601
245 1 0 $a Efficient Models and Learning Strategies for Resource-Constrained Systems.
264 0 $c 2024
300 $a 1 online resource (327 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-12, Section: B.
500 $a Advisor: Huang, Furong.
502 $a Thesis (Ph.D.)--University of Maryland, College Park, 2024.
504 $a Includes bibliographical references
520 $a The last decade has seen sharp improvements in the performance of machine learning (ML) models, but at the cost of vastly increased complexity and size of their underlying architectures. Advances in high-performance computing have enabled researchers to train and deploy models composed of hundreds of billions of parameters. However, harvesting the full utility of large models on smaller clients, such as Internet of Things (IoT) devices, without resorting to external hosting will require a significant reduction of parameters and faster, cheaper inference. In addition to augmenting IoT, efficient models and learning paradigms can reduce energy consumption, encourage technological equity, and are well-suited for deployment in real-world applications which require fast response in low-resource settings.To address these challenges, we introduce multiple, novel strategies for (1) reducing the scale of deep neural networks and (2) faster learning. For the size problem (1), we leverage tools such as tensorization, randomized projections, and locality-sensitive hashing to train on reduced representations of large models without sacrificing performance. For learning efficiency (2), we develop algorithms for cheaper forward passes, accelerated PCA, and asynchronous gradient descent. Several of these methods are tailored for federated learning (FL), a private, distributed learning paradigm where data is decentralized among resource-constrained edge clients. We are exclusively concerned with improving efficiency during training - our techniques do not process pre-trained models or require a device to train over an architecture in its full entirety.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Computer science. $3 573171
650 4 $a Computer engineering. $3 569006
650 4 $a Information technology. $3 559429
653 $a Compression
653 $a Distributed learning
653 $a Efficiency
653 $a Machine learning
653 $a Resource-constrained edge clients
655 7 $a Electronic books. $2 local $3 554714
690 $a 0984
690 $a 0489
690 $a 0464
690 $a 0800
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a University of Maryland, College Park. $b Computer Science. $3 1180862
773 0 $t Dissertations Abstracts International $g 85-12B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31144938 $z click for full text (PQDT)