國立虎尾科技大學 |

Open System Neural Networks.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Open System Neural Networks./
Author:	Hatch, Bradley.
Description:	1 online resource (19 pages)
Notes:	Source: Masters Abstracts International, Volume: 85-12.
Contained By:	Masters Abstracts International85-12.
Subject:	Linguistics. -
Online resource:	click for full text (PQDT)
ISBN:	9798383012642

Open System Neural Networks.
Hatch, Bradley.

Open System Neural Networks. - 1 online resource (19 pages)

Source: Masters Abstracts International, Volume: 85-12.

Thesis (M.Sc.)--Brigham Young University, 2024.

Includes bibliographical references

Recent advances in self-supervised learning have made it possible to reuse information-rich models that have been generally pre-trained on massive amounts of data for other downstream tasks. But the pre-training process can be drastically different from the fine-tuning training process, which can lead to inefficient learning. We address this disconnect in training dynamics by structuring the learning process like an open system in thermodynamics. Open systems can achieve a steady state when low-entropy inputs are converted to high-entropy outputs. We modify the the model and the learning process to mimic this behavior, and attend more to elements of the input sequence that exhibit greater changes in entropy. We call this architecture the Open System Neural Network (OSNN). We show the efficacy of the OSNN on multiple classification datasets with a variety of encoder-only Transformers. We find that the OSNN outperforms nearly all model specific baselines, and achieves a new state-of-the-art result on two classification datasets.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798383012642Subjects--Topical Terms:

557829
Linguistics.
Index Terms--Genre/Form:

554714
Electronic books.

Open System Neural Networks.
LDR:02264ntm a22003497 4500 001 1150150
005 20241022111600.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798383012642
035 $a (MiAaPQ)AAI31097653
035 $a (MiAaPQ)BrighamYoung11243
035 $a AAI31097653
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Hatch, Bradley. $3 1476584
245 1 0 $a Open System Neural Networks.
264 0 $c 2024
300 $a 1 online resource (19 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Masters Abstracts International, Volume: 85-12.
500 $a Advisor: Fulda, Nancy;Farrell, Ryan;Wingate, David;Jenkins, Porter.
502 $a Thesis (M.Sc.)--Brigham Young University, 2024.
504 $a Includes bibliographical references
520 $a Recent advances in self-supervised learning have made it possible to reuse information-rich models that have been generally pre-trained on massive amounts of data for other downstream tasks. But the pre-training process can be drastically different from the fine-tuning training process, which can lead to inefficient learning. We address this disconnect in training dynamics by structuring the learning process like an open system in thermodynamics. Open systems can achieve a steady state when low-entropy inputs are converted to high-entropy outputs. We modify the the model and the learning process to mimic this behavior, and attend more to elements of the input sequence that exhibit greater changes in entropy. We call this architecture the Open System Neural Network (OSNN). We show the efficacy of the OSNN on multiple classification datasets with a variety of encoder-only Transformers. We find that the OSNN outperforms nearly all model specific baselines, and achieves a new state-of-the-art result on two classification datasets.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Linguistics. $3 557829
650 4 $a Sentiment analysis. $3 1403973
650 4 $a Neural networks. $3 1011215
650 4 $a Computer science. $3 573171
655 7 $a Electronic books. $2 local $3 554714
690 $a 0290
690 $a 0800
690 $a 0984
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a Brigham Young University. $3 1187870
773 0 $t Masters Abstracts International $g 85-12.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31097653 $z click for full text (PQDT)