國立虎尾科技大學 |

Small Data Big Insights : = Investigating the Performance of a Small Language Model in Its Ability to Generate English Text.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Small Data Big Insights :/
Reminder of title:	Investigating the Performance of a Small Language Model in Its Ability to Generate English Text.
Author:	Sharma, Tanya.
Description:	1 online resource (48 pages)
Notes:	Source: Masters Abstracts International, Volume: 85-12.
Contained By:	Masters Abstracts International85-12.
Subject:	Statistics. -
Online resource:	click for full text (PQDT)
ISBN:	9798382829753

Small Data Big Insights : = Investigating the Performance of a Small Language Model in Its Ability to Generate English Text.
Sharma, Tanya.

Small Data Big Insights :Investigating the Performance of a Small Language Model in Its Ability to Generate English Text. - 1 online resource (48 pages)

Source: Masters Abstracts International, Volume: 85-12.

Thesis (M.S.)--University of California, Los Angeles, 2024.

Includes bibliographical references

It is widely held that larger language models, trained on vast quantities of text, excel at generating coherent and fluent text. But at the same time, Small Language Models still struggle to produce meaningful text beyond a few words. The specific scale at which these abilities emerge is still not well-defined. Consequently, the lingering question remains: must a model be large-scale to generate coherent text?In this paper we have have trained a small language model on Tiny Stories, a synthetic dataset of short stories. The objective is to study the small language models in their ability to generate coherent and consistent English text. We have performed a comparative study where we have analyzed the convergence of loss and investigated how adjustments to the number of heads, layers, and embedding size affect the generation of English text in Small Language Models.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798382829753Subjects--Topical Terms:

556824
Statistics.
Subjects--Index Terms:

Emergent abilitiesIndex Terms--Genre/Form:

554714
Electronic books.

Small Data Big Insights : = Investigating the Performance of a Small Language Model in Its Ability to Generate English Text.
LDR:02303ntm a22003977 4500 001 1146576
005 20240812064649.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798382829753
035 $a (MiAaPQ)AAI31331338
035 $a AAI31331338
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Sharma, Tanya. $3 1471983
245 1 0 $a Small Data Big Insights : $b Investigating the Performance of a Small Language Model in Its Ability to Generate English Text.
264 0 $c 2024
300 $a 1 online resource (48 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Masters Abstracts International, Volume: 85-12.
500 $a Advisor: Nian Wu, Ying.
502 $a Thesis (M.S.)--University of California, Los Angeles, 2024.
504 $a Includes bibliographical references
520 $a It is widely held that larger language models, trained on vast quantities of text, excel at generating coherent and fluent text. But at the same time, Small Language Models still struggle to produce meaningful text beyond a few words. The specific scale at which these abilities emerge is still not well-defined. Consequently, the lingering question remains: must a model be large-scale to generate coherent text?In this paper we have have trained a small language model on Tiny Stories, a synthetic dataset of short stories. The objective is to study the small language models in their ability to generate coherent and consistent English text. We have performed a comparative study where we have analyzed the convergence of loss and investigated how adjustments to the number of heads, layers, and embedding size affect the generation of English text in Small Language Models.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Statistics. $3 556824
650 4 $a Computer engineering. $3 569006
653 $a Emergent abilities
653 $a Large Language Models
653 $a Natural language processing
653 $a Transformer
653 $a Small Language Models
655 7 $a Electronic books. $2 local $3 554714
690 $a 0463
690 $a 0800
690 $a 0464
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a University of California, Los Angeles. $b Statistics 0891. $3 1183048
773 0 $t Masters Abstracts International $g 85-12.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31331338 $z click for full text (PQDT)