Forecasting Market Closing Direction: A Novel Data Fusion of Deep News Embeddings and Technical Indicators with Gradient Boosting

Accurately predicting stock market movements remains a complex engineering challenge due to inherent non-linearity and volatility. This study addresses this problem by developing a novel artificial intelligence framework that fuses deep learning-based natural language processing with technical analysis for financial forecasting. Its primary engineering application is predicting the daily directional movement of the Tehran Stock Exchange total index, a critical task for automated trading systems and risk management. The core AI contribution is a hybrid architecture that generates contextual text embeddings from Persian news using a Bidirectional Encoder Representations from Transformers (BERT) model and integrates them with technical indicators. These features are processed by a soft-voting ensemble of powerful gradient-boosting algorithms, namely eXtreme Gradient Boosting (XG-Boost) and Light Gradient Boosting Machine (LightGBM). Rigorously validated through an extensive walk-forward testing procedure across ۳۳۱ temporal windows, the model achieved impressive performance metrics: ۸۴.۰% accuracy, ۸۶.۱% F۱ Score, ۸۷.۰% precision, and ۸۵.۲% recall. These notable results demonstrate the significant value of domain-specific natural language processing in non-English financial contexts. Furthermore, this work establishes gradient boosting ensembles as a highly efficient, high-performance alternative to Long Short-Term Memory (LSTM) networks for financial forecasting. The proposed methodology demonstrates robust predictive capability for directional movement forecasting in emerging markets.

کلیدواژه ها:

Stock Market Forecasting ، Persian Natural Language Processing ، Bidirectional Encoder Representations from Transformers Embeddings ، Gradient Boosting Ensemble ، Multi-Modal Data Fusion