國立虎尾科技大學 |

Fine-Tuning BERT for Sentiment Analysis.

紀錄類型:	書目-語言資料,手稿 : Monograph/item
正題名/作者:	Fine-Tuning BERT for Sentiment Analysis./
作者:	Wang, Michelle Lu.
面頁冊數:	1 online resource (83 pages)
附註:	Source: Masters Abstracts International, Volume: 85-09.
Contained By:	Masters Abstracts International85-09.
標題:	Statistics. -
電子資源:	click for full text (PQDT)
ISBN:	9798381952674

Fine-Tuning BERT for Sentiment Analysis.
Wang, Michelle Lu.

Fine-Tuning BERT for Sentiment Analysis. - 1 online resource (83 pages)

Source: Masters Abstracts International, Volume: 85-09.

Thesis (M.S.)--University of California, Los Angeles, 2024.

Includes bibliographical references

The introduction of transformer models have vastly improved the performance of machine learning methods on natural language processing (NLP) tasks. Transformer models use a self-attention mechanism which allow the model to weigh the importance of different words in a sequence when making predictions. They also introduce positional encodings which allow the model to be highly parallelizable, expanding the capacity of data they are able to learn from. This paper explores the fine-tuning of the pre-trained transformer model BERT (Bidirectional Encoder Representations from Transformers) for sentiment analysis on e-commerce reviews. Traditional fine-tuning approaches which involve updating every parameter of a pre-trained model's hundreds of millions of parameters can be inefficient and unnecessary. A parameter-efficient fine-tuning approach is proposed to enhance the pre-trained BERT's performance in discerning between positive and negative sentiments within the diverse user-generate reviews.The methodology begins by preprocessing the data, including text cleaning and tokenization, to prepare the dataset for training. Subsequently, fine-tuning and hyperparameter tuning techniques are applied to the model in order to tailor BERT to the specific qualities of the dataset. Smaller subsets of data are fine-tuned on in order to find optimal hyperparameter settings for fine-tuning the full dataset. Three BERT based models will be explored: BERT$_{BASE}$, RoBERTa$_{BASE}$, and DistilBERT. Each model will be fine-tuned and evaluated in order to find the model which achieves the highest test accuracy rate. The paper will also delve into the obstacles of training with a large dataset, proposing solutions and techniques to circumvent the problems. The findings of this paper show the variations of the models which perform the best greatly depend on the needs of the dataset. The larger dataset analyzed in this paper requires a faster, lighter model in order to process in its entirety. The experiment also explores more robustly optimized and larger models, which yield adequate results using a smaller data subset, but are not suitable for bigger data sets. Hyperparameter settings are shown to affect the performance of the model, but not impact the model in any distinct patterns. The exception to this is the number of epochs the data is trained for, which almost always positively influences model accuracy rates.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798381952674Subjects--Topical Terms:

556824
Statistics.
Subjects--Index Terms:

Machine learningIndex Terms--Genre/Form:

554714
Electronic books.

Fine-Tuning BERT for Sentiment Analysis.
LDR:03673ntm a22003617 4500 001 1146462
005 20240812064622.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798381952674
035 $a (MiAaPQ)AAI30995389
035 $a AAI30995389
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Wang, Michelle Lu. $3 1471851
245 1 0 $a Fine-Tuning BERT for Sentiment Analysis.
264 0 $c 2024
300 $a 1 online resource (83 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Masters Abstracts International, Volume: 85-09.
500 $a Advisor: Wu, Yingnian.
502 $a Thesis (M.S.)--University of California, Los Angeles, 2024.
504 $a Includes bibliographical references
520 $a The introduction of transformer models have vastly improved the performance of machine learning methods on natural language processing (NLP) tasks. Transformer models use a self-attention mechanism which allow the model to weigh the importance of different words in a sequence when making predictions. They also introduce positional encodings which allow the model to be highly parallelizable, expanding the capacity of data they are able to learn from. This paper explores the fine-tuning of the pre-trained transformer model BERT (Bidirectional Encoder Representations from Transformers) for sentiment analysis on e-commerce reviews. Traditional fine-tuning approaches which involve updating every parameter of a pre-trained model's hundreds of millions of parameters can be inefficient and unnecessary. A parameter-efficient fine-tuning approach is proposed to enhance the pre-trained BERT's performance in discerning between positive and negative sentiments within the diverse user-generate reviews.The methodology begins by preprocessing the data, including text cleaning and tokenization, to prepare the dataset for training. Subsequently, fine-tuning and hyperparameter tuning techniques are applied to the model in order to tailor BERT to the specific qualities of the dataset. Smaller subsets of data are fine-tuned on in order to find optimal hyperparameter settings for fine-tuning the full dataset. Three BERT based models will be explored: BERT$_{BASE}$, RoBERTa$_{BASE}$, and DistilBERT. Each model will be fine-tuned and evaluated in order to find the model which achieves the highest test accuracy rate. The paper will also delve into the obstacles of training with a large dataset, proposing solutions and techniques to circumvent the problems. The findings of this paper show the variations of the models which perform the best greatly depend on the needs of the dataset. The larger dataset analyzed in this paper requires a faster, lighter model in order to process in its entirety. The experiment also explores more robustly optimized and larger models, which yield adequate results using a smaller data subset, but are not suitable for bigger data sets. Hyperparameter settings are shown to affect the performance of the model, but not impact the model in any distinct patterns. The exception to this is the number of epochs the data is trained for, which almost always positively influences model accuracy rates.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Statistics. $3 556824
653 $a Machine learning
653 $a Tokenization
653 $a Fine-tuning approaches
655 7 $a Electronic books. $2 local $3 554714
690 $a 0463
690 $a 0800
710 2 $a University of California, Los Angeles. $b Statistics 0891. $3 1183048
710 2 $a ProQuest Information and Learning Co. $3 1178819
773 0 $t Masters Abstracts International $g 85-09.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30995389 $z click for full text (PQDT)