Novel Insights on the Comparative Study Between LSTM and Transformer Models for Financial Time Series Prediction

Abstract

This study investigates the impact of deep learning model architectures and optimization techniques on financial time series forecasting, with a focus on Transformer-based models and Long Short-Term Memory networks. Optimization methods such as Adaptive Moment Estimation and Adaptive Moment Estimation with Infinity Norm play a critical role in enhancing training efficiency and prediction accuracy, while architectural decisions determine each model’s ability to capture both short-term and long-term dependencies in sequential data. Using datasets including stock prices from Microsoft and NVIDIA, the models are evaluated based on metrics such as Mean Absolute Percentage Error, Root Mean Squared Error, prediction variance, and training speed. The results demonstrate the complementary strengths of Transformers and Long Short-Term Memory Networks, underscoring the importance of tailored architectures and optimization strategies in deep learning-based financial forecasting.