國立虎尾科技大學 |

Faster, Incentivized, and Efficient Federated Learning : = Theory and Applications.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Faster, Incentivized, and Efficient Federated Learning :/
Reminder of title:	Theory and Applications.
Author:	Cho, Yae Jee.
Description:	1 online resource (241 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-11, Section: B.
Contained By:	Dissertations Abstracts International85-11B.
Subject:	Computer science. -
Online resource:	click for full text (PQDT)
ISBN:	9798382761572

Faster, Incentivized, and Efficient Federated Learning : = Theory and Applications.
Cho, Yae Jee.

Faster, Incentivized, and Efficient Federated Learning :Theory and Applications. - 1 online resource (241 pages)

Source: Dissertations Abstracts International, Volume: 85-11, Section: B.

Thesis (Ph.D.)--Carnegie Mellon University, 2024.

Includes bibliographical references

Artificial intelligence (AI) is becoming increasingly ubiquitous with a plethora of applications such as recommendation systems, image/video generation, or chatbots changing our modern society. The majority of these AI based applications are based on centralized machine learning (ML). In centralized ML, a large amount of data is collected at a central server to train a large ML model with hundreds of billions of parameters using massive amount of system resources (e.g., CPUs, GPUs). Despite the success of such pipeline for training models in the centralized setting, however, centralized ML has critical limitations. First, data collection can raise serious privacy concerns due to the desired data often being personal data such as medical histories or financial data and users opting out from sharing their data. Second, the exponentially increasing economic and environmental cost for training colossal models in the centralized setting raises concerns for sustainability. Such major drawbacks of centralized ML calls for a new paradigm that can shift the server-based ML pipeline to the edge, where data collection as well as computation happens on-device at the edge client. Moreover, decentralizing ML to the edge can facilitate democratizing the benefits of ML by allowing better personalized models for individual users. Federated learning (FL) is a well-known method that achieves such decentralization of ML where clients (e.g., mobile phones, hospitals, banks) locally train the server ML model(s) on their private data and send only the local gradient updates to the server so that the server can update its model with the aggregation of these local updates. The clients can also privately further fine-tune the server ML model to make personalized models to their data. In this thesis, we aim to address the three main challenges that arise in FL which does not exist in the conventional centralized ML setting: i) clients' limited communication and availability, ii) clients' data heterogeneity, and iii) clients' limited and heterogeneous training resources. We address each challenge with three main research contributions respectively i) faster FL, ii) incentivized FL, and iii) efficient FL. First, we investigate theoretically and empirically how biased client selection and cyclic client participation in FL under limited client communication and availability can improve the convergence of FL for faster FL. Second, we delve into the question of whether clients have incentives to join FL in the first place under client data heterogeneity and how disincentivization of clients can sabotage the training process of FL. We propose a framework to improve the fraction of incentivized clients by incentive-aware weighted aggregation of clients' local updates for incentivized FL and also propose a communication-efficient personalized FL framework to further incentivize clients. Lastly, we propose two novel methods, federated ensemble transfer and heterogeneous LoRA (low-rank approximation) to train larger server models, including foundation models, at the edge with significantly smaller number of parameters under clients' limited and heterogeneous training resources. We also look into efficient FL from the perspective of labels where clients have only a few labels in their local data.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798382761572Subjects--Topical Terms:

573171
Computer science.
Subjects--Index Terms:

ApplicationsIndex Terms--Genre/Form:

554714
Electronic books.

Faster, Incentivized, and Efficient Federated Learning : = Theory and Applications.
LDR:04688ntm a22003977 4500 001 1146557
005 20240812064644.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798382761572
035 $a (MiAaPQ)AAI31300766
035 $a AAI31300766
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Cho, Yae Jee. $3 1471964
245 1 0 $a Faster, Incentivized, and Efficient Federated Learning : $b Theory and Applications.
264 0 $c 2024
300 $a 1 online resource (241 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-11, Section: B.
500 $a Advisor: Joshi, Gauri.
502 $a Thesis (Ph.D.)--Carnegie Mellon University, 2024.
504 $a Includes bibliographical references
520 $a Artificial intelligence (AI) is becoming increasingly ubiquitous with a plethora of applications such as recommendation systems, image/video generation, or chatbots changing our modern society. The majority of these AI based applications are based on centralized machine learning (ML). In centralized ML, a large amount of data is collected at a central server to train a large ML model with hundreds of billions of parameters using massive amount of system resources (e.g., CPUs, GPUs). Despite the success of such pipeline for training models in the centralized setting, however, centralized ML has critical limitations. First, data collection can raise serious privacy concerns due to the desired data often being personal data such as medical histories or financial data and users opting out from sharing their data. Second, the exponentially increasing economic and environmental cost for training colossal models in the centralized setting raises concerns for sustainability. Such major drawbacks of centralized ML calls for a new paradigm that can shift the server-based ML pipeline to the edge, where data collection as well as computation happens on-device at the edge client. Moreover, decentralizing ML to the edge can facilitate democratizing the benefits of ML by allowing better personalized models for individual users. Federated learning (FL) is a well-known method that achieves such decentralization of ML where clients (e.g., mobile phones, hospitals, banks) locally train the server ML model(s) on their private data and send only the local gradient updates to the server so that the server can update its model with the aggregation of these local updates. The clients can also privately further fine-tune the server ML model to make personalized models to their data. In this thesis, we aim to address the three main challenges that arise in FL which does not exist in the conventional centralized ML setting: i) clients' limited communication and availability, ii) clients' data heterogeneity, and iii) clients' limited and heterogeneous training resources. We address each challenge with three main research contributions respectively i) faster FL, ii) incentivized FL, and iii) efficient FL. First, we investigate theoretically and empirically how biased client selection and cyclic client participation in FL under limited client communication and availability can improve the convergence of FL for faster FL. Second, we delve into the question of whether clients have incentives to join FL in the first place under client data heterogeneity and how disincentivization of clients can sabotage the training process of FL. We propose a framework to improve the fraction of incentivized clients by incentive-aware weighted aggregation of clients' local updates for incentivized FL and also propose a communication-efficient personalized FL framework to further incentivize clients. Lastly, we propose two novel methods, federated ensemble transfer and heterogeneous LoRA (low-rank approximation) to train larger server models, including foundation models, at the edge with significantly smaller number of parameters under clients' limited and heterogeneous training resources. We also look into efficient FL from the perspective of labels where clients have only a few labels in their local data.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Computer science. $3 573171
650 4 $a Computer engineering. $3 569006
650 4 $a Electrical engineering. $3 596380
653 $a Applications
653 $a Federated learning
653 $a Machine learning
653 $a Environmental cost
655 7 $a Electronic books. $2 local $3 554714
690 $a 0984
690 $a 0544
690 $a 0464
690 $a 0800
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a Carnegie Mellon University. $b Electrical and Computer Engineering. $3 1182305
773 0 $t Dissertations Abstracts International $g 85-11B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31300766 $z click for full text (PQDT)