國立虎尾科技大學 |

Enhancing Transformer-Based Architectures With Temporal Hierarchies for Time-Series Forecasting.

Record Type:	Language materials, manuscript : Monograph/item
Title/Author:	Enhancing Transformer-Based Architectures With Temporal Hierarchies for Time-Series Forecasting./
Author:	Bachyrycz, Emma.
Description:	1 online resource (108 pages)
Notes:	Source: Masters Abstracts International, Volume: 85-12.
Contained By:	Masters Abstracts International85-12.
Subject:	Systems science. -
Online resource:	click for full text (PQDT)
ISBN:	9798383099575

Enhancing Transformer-Based Architectures With Temporal Hierarchies for Time-Series Forecasting.
Bachyrycz, Emma.

Enhancing Transformer-Based Architectures With Temporal Hierarchies for Time-Series Forecasting. - 1 online resource (108 pages)

Source: Masters Abstracts International, Volume: 85-12.

Thesis (M.S.)--State University of New York at Binghamton, 2024.

Includes bibliographical references

This research proposes an enhanced transformer model for time-series forecasting to align the various levels of decision-making. Transformer models were initially proposed in 2017 for natural language processing and have gained popularity since, notably with the release of ChatGPT. This architecture is unique with its parallel multi-head attention mechanism that addresses the concerns of the popular deep learning model recurrent neural networks. Transformer mechanisms can handle large datasets without experiencing the vanishing gradient problem, aligning with the rise of the Industry 4.0 Revolution and Big Data. In this research we explore the opportunities of transformer models in time-series forecasting, offering enhancements using temporal hierarchies. This involves aggregating the data at different temporal scales, such as daily monthly, and yearly, and incorporating the hierarchical levels into the model's framework. By training the model on recurrent structures at different frequencies, our preliminary results encourage accurate, dynamic, and robust forecasting of the dataset that outstands alternatives. The various temporal hierarchies represent the levels of decision-making within an organization. The monthly dataset aligns with operational decisions frequently made by lower management. The quarterly dataset is the tactical level, which incorporates the medium-term goals of a company and is made by middle management. Finally, the yearly dataset is for the strategic-level decisions made by executives. The results show the model combining the monthly, quarterly, and yearly temporal hierarchies performed with the most accuracy, excelling above the benchmark models. With three attention heads and a forecasting horizon of two years, the model performed with an MSE of 1.741, sMAPE of 0.296%, and RMSE of 1.319. Under the same forecasting parameters, the second-highest-performing model is the forecast combining operational and tactical levels or monthly and quarterly data. This model performed with an MSE of 1.885, sMAPE of 0.329%, and RSME of 1.373.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2024

Mode of access: World Wide Web

ISBN: 9798383099575Subjects--Topical Terms:

1148479
Systems science.
Subjects--Index Terms:

Decision-makingIndex Terms--Genre/Form:

554714
Electronic books.

Enhancing Transformer-Based Architectures With Temporal Hierarchies for Time-Series Forecasting.
LDR:03436ntm a22003857 4500 001 1150192
005 20241022111611.5
006 m o d
007 cr bn ---uuuuu
008 250605s2024 xx obm 000 0 eng d
020 $a 9798383099575
035 $a (MiAaPQ)AAI31244375
035 $a AAI31244375
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Bachyrycz, Emma. $3 1476633
245 1 0 $a Enhancing Transformer-Based Architectures With Temporal Hierarchies for Time-Series Forecasting.
264 0 $c 2024
300 $a 1 online resource (108 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Masters Abstracts International, Volume: 85-12.
500 $a Advisor: Wang, Yong.
502 $a Thesis (M.S.)--State University of New York at Binghamton, 2024.
504 $a Includes bibliographical references
520 $a This research proposes an enhanced transformer model for time-series forecasting to align the various levels of decision-making. Transformer models were initially proposed in 2017 for natural language processing and have gained popularity since, notably with the release of ChatGPT. This architecture is unique with its parallel multi-head attention mechanism that addresses the concerns of the popular deep learning model recurrent neural networks. Transformer mechanisms can handle large datasets without experiencing the vanishing gradient problem, aligning with the rise of the Industry 4.0 Revolution and Big Data. In this research we explore the opportunities of transformer models in time-series forecasting, offering enhancements using temporal hierarchies. This involves aggregating the data at different temporal scales, such as daily monthly, and yearly, and incorporating the hierarchical levels into the model's framework. By training the model on recurrent structures at different frequencies, our preliminary results encourage accurate, dynamic, and robust forecasting of the dataset that outstands alternatives. The various temporal hierarchies represent the levels of decision-making within an organization. The monthly dataset aligns with operational decisions frequently made by lower management. The quarterly dataset is the tactical level, which incorporates the medium-term goals of a company and is made by middle management. Finally, the yearly dataset is for the strategic-level decisions made by executives. The results show the model combining the monthly, quarterly, and yearly temporal hierarchies performed with the most accuracy, excelling above the benchmark models. With three attention heads and a forecasting horizon of two years, the model performed with an MSE of 1.741, sMAPE of 0.296%, and RMSE of 1.319. Under the same forecasting parameters, the second-highest-performing model is the forecast combining operational and tactical levels or monthly and quarterly data. This model performed with an MSE of 1.885, sMAPE of 0.329%, and RSME of 1.373.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2024
538 $a Mode of access: World Wide Web
650 4 $a Systems science. $3 1148479
650 4 $a Industrial engineering. $3 679492
653 $a Decision-making
653 $a Temporal hierarchies
653 $a Time-series
653 $a Transformer model
655 7 $a Electronic books. $2 local $3 554714
690 $a 0790
690 $a 0800
690 $a 0546
710 2 $a ProQuest Information and Learning Co. $3 1178819
710 2 $a State University of New York at Binghamton. $b Systems Science Industrial Engineering. $3 1182293
773 0 $t Masters Abstracts International $g 85-12.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31244375 $z click for full text (PQDT)