國立虎尾科技大學 |

Continual Learning With Language Models.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Continual Learning With Language Models./
Author:	Ke, Zixuan.
Description:	1 online resource (162 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 86-01, Section: B.
Contained By:	Dissertations Abstracts International86-01B.
Subject:	Linguistics. -
Online resource:	click for full text (PQDT)
ISBN:	9798382952048

Continual Learning With Language Models.
Ke, Zixuan.

Continual Learning With Language Models. - 1 online resource (162 pages)

Source: Dissertations Abstracts International, Volume: 86-01, Section: B.

Thesis (Ph.D.)--University of Illinois at Chicago, 2024.

Includes bibliographical references

The essence of human intelligence lies in its ability to learn continuously, accumulating past knowledge to aid in future learning and problem-solving endeavors. In contrast, the current machine learning paradigm often operates in isolation, lacking the capacity for continual learning and adaptation. This deficiency becomes apparent in the face of rapidly evolving artificial intelligence (AI) technologies, particularly large language models (LLMs), where incremental training remains a challenge. Continual learning (CL), also known as lifelong learning, is indispensable for truly intelligent systems, especially in dynamic environments where constant adaptation is necessary. This dissertation explores recent advancements in continual learning algorithms within the framework of language models. We first introduce the settings, challenges, and general approaches of CL. We then delve into our efforts to achieve both catastrophic forgetting (CF) mitigation and knowledge transfer (KT), and how we apply CL to different stages of language model development, including pre-training and end-task adaptation. With the aid of continual learning, the performance of language models is greatly improved. Finally, we will discuss the opportunities for AI autonomy and open-world continual learning.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798382952048Subjects--Topical Terms:

557829
Linguistics.
Subjects--Index Terms:

Continual learningIndex Terms--Genre/Form:

554714
Electronic books.

Continual Learning With Language Models.
LDR:02716ntm a22004217 4500 001 1147905
005 20240916075427.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798382952048
035 $a (MiAaPQ)AAI31496164
035 $a (MiAaPQ)0799vireo386Ke
035 $a AAI31496164
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Ke, Zixuan. $3 1473748
245 1 0 $a Continual Learning With Language Models.
264 0 $c 2024
300 $a 1 online resource (162 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 86-01, Section: B.
500 $a Advisor: Liu, Bing.
502 $a Thesis (Ph.D.)--University of Illinois at Chicago, 2024.
504 $a Includes bibliographical references
520 $a The essence of human intelligence lies in its ability to learn continuously, accumulating past knowledge to aid in future learning and problem-solving endeavors. In contrast, the current machine learning paradigm often operates in isolation, lacking the capacity for continual learning and adaptation. This deficiency becomes apparent in the face of rapidly evolving artificial intelligence (AI) technologies, particularly large language models (LLMs), where incremental training remains a challenge. Continual learning (CL), also known as lifelong learning, is indispensable for truly intelligent systems, especially in dynamic environments where constant adaptation is necessary. This dissertation explores recent advancements in continual learning algorithms within the framework of language models. We first introduce the settings, challenges, and general approaches of CL. We then delve into our efforts to achieve both catastrophic forgetting (CF) mitigation and knowledge transfer (KT), and how we apply CL to different stages of language model development, including pre-training and end-task adaptation. With the aid of continual learning, the performance of language models is greatly improved. Finally, we will discuss the opportunities for AI autonomy and open-world continual learning.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Linguistics. $3 557829
650 4 $a Computer science. $3 573171
650 4 $a Information technology. $3 559429
653 $a Continual learning
653 $a Language models
653 $a Natural language processing
653 $a Machine learning
653 $a Knowledge transfer
655 7 $a Electronic books. $2 local $3 554714
690 $a 0489
690 $a 0984
690 $a 0800
690 $a 0290
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a University of Illinois at Chicago. $b Computer Science. $3 1467155
773 0 $t Dissertations Abstracts International $g 86-01B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31496164 $z click for full text (PQDT)