國立虎尾科技大學 |

Transformer-Enhanced Text Classification in Cybersecurity : = GPT-Augmented Synthetic Data Generation, BERT-Based Semantic Encoding, and Multiclass Analysis.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Transformer-Enhanced Text Classification in Cybersecurity :/
Reminder of title:	GPT-Augmented Synthetic Data Generation, BERT-Based Semantic Encoding, and Multiclass Analysis.
Author:	Houston, Robert A.
Description:	1 online resource (201 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-06, Section: B.
Contained By:	Dissertations Abstracts International85-06B.
Subject:	Computer engineering. -
Online resource:	click for full text (PQDT)
ISBN:	9798381111774

Transformer-Enhanced Text Classification in Cybersecurity : = GPT-Augmented Synthetic Data Generation, BERT-Based Semantic Encoding, and Multiclass Analysis.
Houston, Robert A.

Transformer-Enhanced Text Classification in Cybersecurity :GPT-Augmented Synthetic Data Generation, BERT-Based Semantic Encoding, and Multiclass Analysis. - 1 online resource (201 pages)

Source: Dissertations Abstracts International, Volume: 85-06, Section: B.

Thesis (D.Engr.)--The George Washington University, 2024.

Includes bibliographical references

This research presents an examination and findings based on an investigation into the use of Large Language Models (LLMs) for generative and encoding tasks in the domain of cybersecurity. The work presented herein demonstrates some critical uses of transformer-derived deep learning model architectures that use Generative Pretrained Transformer Model 2 (GPT-2) to generate synthetic data to balance otherwise imbalanced data sets for multi-class classification for enhancing cybersecurity and families of Bidirectional Encoder Representations from Transformers (BERT) used for word embeddings. Three factors link these research areas. First, the BERT embeddings are derived from a synthetically balanced dataset produced by GPT-2 generative capabilities. Second, both BERT and GPT-2 models are pretrained models that are subsequently fine-tuned using each minority class of the data set. Finally, BERT and GPT-2 are derived from the transformers model introduced by the seminal paper, Attention Is All You Need (Vaswani et al., 2017). In this research, synthetic data sets and the embedding models produced using transformer models are evaluated using various traditional Machine Learning (ML) models using a novel weighted aggregation of F1 score developed to account for the disparate risk inherent with different classification classes for cybersecurity applications.Another aspect of this research centers on using publicly licensed models with open-source base-training weights. The objective here is to analyze lightweight LLMs that are both deployable in computationally constrained environments and available for use in secure environments with no need to pass data to public cloud enclaves via commercial API calls to monetized foundational models.This research addresses essential needs relevant to using AI models in cybersecurity applications. The primary goals address the need for balanced, high-quality training data sets and semantically aware Natural Language Processing (NLP) vectorization methods for ML models. As a backdrop, there is an emphasis on the use of lightweight democratized models that are not monetized and are widely publicly available with open-source model weights.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798381111774Subjects--Topical Terms:

569006
Computer engineering.
Subjects--Index Terms:

Generative Pretrained TransformersIndex Terms--Genre/Form:

554714
Electronic books.

Transformer-Enhanced Text Classification in Cybersecurity : = GPT-Augmented Synthetic Data Generation, BERT-Based Semantic Encoding, and Multiclass Analysis.
LDR:03715ntm a22003977 4500 001 1143911
005 20240517105025.5
006 m o d
007 cr mn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798381111774
035 $a (MiAaPQ)AAI30815867
035 $a AAI30815867
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Houston, Robert A. $3 1468721
245 1 0 $a Transformer-Enhanced Text Classification in Cybersecurity : $b GPT-Augmented Synthetic Data Generation, BERT-Based Semantic Encoding, and Multiclass Analysis.
264 0 $c 2024
300 $a 1 online resource (201 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-06, Section: B.
500 $a Advisor: Rivera, Domingo.
502 $a Thesis (D.Engr.)--The George Washington University, 2024.
504 $a Includes bibliographical references
520 $a This research presents an examination and findings based on an investigation into the use of Large Language Models (LLMs) for generative and encoding tasks in the domain of cybersecurity. The work presented herein demonstrates some critical uses of transformer-derived deep learning model architectures that use Generative Pretrained Transformer Model 2 (GPT-2) to generate synthetic data to balance otherwise imbalanced data sets for multi-class classification for enhancing cybersecurity and families of Bidirectional Encoder Representations from Transformers (BERT) used for word embeddings. Three factors link these research areas. First, the BERT embeddings are derived from a synthetically balanced dataset produced by GPT-2 generative capabilities. Second, both BERT and GPT-2 models are pretrained models that are subsequently fine-tuned using each minority class of the data set. Finally, BERT and GPT-2 are derived from the transformers model introduced by the seminal paper, Attention Is All You Need (Vaswani et al., 2017). In this research, synthetic data sets and the embedding models produced using transformer models are evaluated using various traditional Machine Learning (ML) models using a novel weighted aggregation of F1 score developed to account for the disparate risk inherent with different classification classes for cybersecurity applications.Another aspect of this research centers on using publicly licensed models with open-source base-training weights. The objective here is to analyze lightweight LLMs that are both deployable in computationally constrained environments and available for use in secure environments with no need to pass data to public cloud enclaves via commercial API calls to monetized foundational models.This research addresses essential needs relevant to using AI models in cybersecurity applications. The primary goals address the need for balanced, high-quality training data sets and semantically aware Natural Language Processing (NLP) vectorization methods for ML models. As a backdrop, there is an emphasis on the use of lightweight democratized models that are not monetized and are widely publicly available with open-source model weights.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Computer engineering. $3 569006
650 4 $a Engineering. $3 561152
653 $a Generative Pretrained Transformers
653 $a Large Language Models
653 $a Machine Learning
653 $a Natural Language Processing
653 $a Synthetic data augmentation
655 7 $a Electronic books. $2 local $3 554714
690 $a 0800
690 $a 0464
690 $a 0537
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a The George Washington University. $b Cybersecurity Analytics. $3 1467608
773 0 $t Dissertations Abstracts International $g 85-06B.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30815867 $z click for full text (PQDT)