Statistical Accuracy Versus Trading Utility: A Multi-Horizon Evaluation of Generative-Ai-Augmented Deep Learning on the Kse-100 Index

Zaka Ullah Saif; Saira Gillani; Anila Gulzar Toor

Authors

Zaka Ullah Saif Faculty of Information Technology and Computer Science, University of Central Punjab, Lahore, Pakistan
Saira Gillani Faculty of Information Technology and Computer Science, University of Central Punjab, Lahore, Pakistan
Anila Gulzar Toor Info Science AI, LLC Seattle, WA

Keywords:

Algorithmic Trading, Generative AI, Variational Autoencoder, Deep Learning (DL), PatchTST, Extreme Gradient Boosting (XGBoost), Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (BiLSTM), Gated Recurrent Unit (GRU), Walk-Forward Validation, Karachi Stock Exchange (KSE-100), Risk-Adjusted Return, Forecast Evaluation

Abstract

Forecasting financial markets is increasingly approached through deep learning, with recent work proposing generative augmentation as a means to improve predictive accuracy under the high noise and non-stationarity that characterize emerging-market equities. This study evaluates whether such accuracy gains translate into trading utility on the Karachi Stock Exchange 100 (KSE-100) index, using daily data from February 2008 to February 2024 (4,176 observations) spanning three crisis episodes. Eleven modeling configurations: Ridge regression, Random Forest, XGBoost, LSTM, GRU, BiLSTM, PatchTST, and a β-variational-autoencoder augmentation of each deep-learning backbone are compared under an identical expanding-window walk-forward protocol with thirteen non-overlapping test folds, four forecast horizons (1, 5, 10, and 20 days) and a cost-aware backtest applying 5 bps fees plus 2 bps slippage. Performance is reported on both statistical (MAE, RMSE, Diebold–Mariano) and economic (Sharpe ratio, CAGR, maximum drawdown, turnover) criteria. Prior evaluations of GenAI augmentation in finance assess a single backbone at a single horizon without cost-aware trading validation. No such multi-horizon, multi-backbone evaluation has previously been reported for the KSE-100. PatchTST+GenAI achieves the lowest mean MAE (0.0450) and the lowest mean RMSE (0.0566) across horizons, with Diebold–Mariano tests confirming statistically significant accuracy gains over its unaugmented counterpart at horizons of 5, 10 and 20 days. However, this accuracy advantage does not translate to trading utility: PatchTST+GenAI records a Sharpe ratio of −0.61 and a maximum drawdown of 96%. The only profitable strategy is the unaugmented LSTM (Sharpe:0.64, CAGR:15.6%, maximum drawdown: 40%), which ranks fourth on mean MAE, holds statistically significant positive directional accuracy (51.95%, p = 0.032), remains profitable after costs of up to 30 bps and is the only strategy whose positive Sharpe ratio is statistically distinguishable from zero by block bootstrap (p = 0.008, 95% CI: [0.16, 1.14]). An inverse-MAE ensemble of the leading forecasters underperforms the LSTM on every trading metric. The findings document a pronounced and statistically verified decoupling between statistical accuracy and economic utility in emerging-market forecasting and motivate decision-aware loss functions for trading-oriented model development.

References

R. Cont, “Empirical properties of asset returns: stylized facts and statistical issues,” Quant. Financ., vol. 1, no. 2, pp. 223–236, 2001, doi: 10.1080/713665670.

B. B. Mandelbrot, “The variation of certain speculative prices,” Fractals Scaling Financ., pp. 371–418, 1997, doi: 10.1007/978-1-4757-2763-0_14.

M. M. López de Prado, “Advances in financial machine learning,” p. 366, 2018, Accessed: Jun. 07, 2026. [Online]. Available: https://www.wiley.com/en-us/advances-in-financial-machine-learning-p-9781119482086

G. Elliott and A. Timmermann, “Economic Forecasting,” Econ. Books, 2016, Accessed: Jun. 07, 2026. [Online]. Available: https://ideas.repec.org/b/pup/pbooks/10740.html

Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, Jayant Kalagnanam, “A Time Series is Worth 64 Words: Long-term Forecasting with Transformers,” arXiv:2211.14730, 2023, [Online]. Available: https://arxiv.org/abs/2211.14730

D. P. Kingma and M. Welling, “Auto-encoding variational Bayes,” arXiv:1312.6114, 2013, [Online]. Available: https://arxiv.org/abs/1312.6114

Irina Higgins, Loic Matthey, “beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework,” Open Rev., 2017, [Online]. Available: https://openreview.net/forum?id=Sy2fzU9gl

Geoffrey Hinton, Oriol Vinyals, Jeff Dean, “Distilling the Knowledge in a Neural Network,” arXiv:1503.02531, 2015, [Online]. Available: https://arxiv.org/abs/1503.02531

“Box, G.E.P. and Jenkins, G.M. (1970) Time Series Analysis Forecasting and Control. Holden-Day, San Francisco. - References - Scientific Research Publishing.” Accessed: May 05, 2024. [Online]. Available: https://www.scirp.org/reference/referencespapers?referenceid=2087370

R. F. Engle, “Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation,” Econometrica, vol. 50, no. 4, p. 987, Jul. 1982, doi: 10.2307/1912773.

T. Bollerslev, “Generalized autoregressive conditional heteroskedasticity,” J. Econom., vol. 31, no. 3, pp. 307–327, 1986, doi: 10.1016/0304-4076(86)90063-1.

J. D. Hamilton, “A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle,” Econometrica, vol. 57, no. 2, p. 357, Mar. 1989, doi: 10.2307/1912559.

C. W. J. Granger, Roselyne Joyeux, “An Introduction To Long-Memory Time Series Models And Fractional Differencing,” J. Time Ser. Anal., 1980, [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-9892.1980.tb00297.x

R. T. Baillie, T. Bollerslev, and H. O. Mikkelsen, “Fractionally integrated generalized autoregressive conditional heteroskedasticity,” J. Econom., vol. 74, no. 1, pp. 3–30, 1996, doi: 10.1016/S0304-4076(95)01749-6.

S. J. Taylor, “Modeling Stochastic Volatility: A Review And Comparative Study,” Math. Financ., vol. 4, no. 2, pp. 183–204, Apr. 1994, doi: 10.1111/J.1467-9965.1994.Tb00057.X;Subpage:String:Abstract;Website:Website:Pericles;Journal:Journal:14679965;Wgroup:String:Publication.

A. E. Hoerl and R. W. Kennard, “Ridge Regression: Biased Estimation for Nonorthogonal Problems,” Technometrics, vol. 12, no. 1, pp. 55–67, 1970, doi: 10.1080/00401706.1970.10488634.

Robert Tibshirani, “Regression Shrinkage and Selection Via the Lasso Free,” J. R. Stat. Soc. Ser. B, vol. 58, no. 1, pp. 267–288, 1996, doi: https://doi.org/10.1111/j.2517-6161.1996.tb02080.x.

Leo Breiman, “Random Forests,” Mach. Learn., vol. 45, pp. 5–32, 2001, [Online]. Available: https://link.springer.com/article/10.1023/A:1010933404324

T. Chen and C. Guestrin, “XGBoost: A Scalable Tree Boosting System,” Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., vol. 13-17-August-2016, pp. 785–794, Mar. 2016, doi: 10.1145/2939672.2939785.

Shihao Gu, Bryan T. Kelly, “Empirical Asset Pricing via Machine Learning,” SSRN Electron. J., 2018, [Online]. Available: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3159577

D. H. Bailey, J. Borwein, M. Lopez de Prado, and Q. J. Zhu, “Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance,” SSRN Electron. J., Apr. 2014, doi: 10.2139/SSRN.2308659.

Christoph Bergmeir, José M. Benítez, “On the use of cross-validation for time series predictor evaluation,” Inf. Sci. (Ny)., pp. 192–213, 2012, doi: https://doi.org/10.1016/j.ins.2011.12.028.

S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, Nov. 1997, doi: 10.1162/NECO.1997.9.8.1735.

Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio, “Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation,” arXiv:1406.1078, 2014, [Online]. Available: https://arxiv.org/abs/1406.1078

G. Liu and J. Guo, “Bidirectional LSTM with attention mechanism and convolutional layer for text classification,” Neurocomputing, vol. 337, pp. 325–338, Apr. 2019, doi: 10.1016/J.NEUCOM.2019.01.078.

T. Fischer and C. Krauss, “Deep learning with long short-term memory networks for financial market predictions,” Eur. J. Oper. Res., vol. 270, no. 2, pp. 654–669, Oct. 2018, doi: 10.1016/J.EJOR.2017.11.054.

Ha Young Kim, Chang Hyun Won, “Forecasting the volatility of stock price index: A hybrid model integrating LSTM with multiple GARCH-type models,” Expert Syst. Appl., vol. 103, pp. 25–37, 2018, [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0957417418301416

I. P. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, “Attention Is All You Need,” arXiv:1706.03762, 2017, doi: https://doi.org/10.48550/arXiv.1706.03762.

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, Wancai Zhang, “Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting,” arXiv:2012.07436, 2020, [Online]. Available: https://arxiv.org/abs/2012.07436

Haixu Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long, “Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting,” arXiv:2106.13008, 2021, [Online]. Available: https://arxiv.org/abs/2106.13008

Ailing Zeng, Muxi Chen, Lei Zhang, Qiang Xu, “Are Transformers Effective for Time Series Forecasting?,” arXiv:2205.13504, 2022, [Online]. Available: https://arxiv.org/abs/2205.13504

S. A. Bryan Lim, “Temporal Fusion Transformers for interpretable multi-horizon time series forecasting,” Int. J. Forecast., vol. 37, no. 4, pp. 1748–1764, 2021, [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0169207021000637

C. E. Chang, W. A. Nelson, and H. D. Witte, “Do green mutual funds perform well?,” Manag. Res. Rev., vol. 35, no. 8, pp. 693–708, Jul. 2012, doi: 10.1108/01409171211247695.

F. A. Sortino, L. N. Price, F. A. Sortino, and L. N. Price, “Performance Measurement in a Downside Risk Framework,” J. Invest., vol. 3, no. 3, pp. 59–64, Aug. 1994, doi: 10.3905/JOI.3.3.59.

M. Magdon-Ismail and A. F. Atiya, “Maximum Drawdown,” 2004. Accessed: Jun. 07, 2026. [Online]. Available: https://papers.ssrn.com/abstract=874069

A. Ang and G. Bekaert, “Regime Switches in Interest Rates,” J. Bus. Econ. Stat., vol. 20, no. 2, pp. 163–182, 2002, doi: 10.1198/073500102317351930.

Magnus Wiese, Robert Knobloch, Ralf Korn, Peter Kretschmer, “Quant GANs: Deep Generation of Financial Time Series,” arXiv:1907.06673, 2019, [Online]. Available: https://arxiv.org/abs/1907.06673

Wei Bao, Jun Yue, Yulei Rao, “A deep learning framework for financial time series using stacked autoencoders and long-short term memory,” PLoS One, 2017, doi: https://doi.org/10.1371/journal.pone.0180944.

Marco Schreyer, Timur Sattarov, Damian Borth, Andreas Dengel, Bernd Reimer, “Detection of Anomalies in Large Scale Accounting Data using Deep Autoencoder Networks,” arXiv:1709.05254, 2018, [Online]. Available: https://arxiv.org/abs/1709.05254

Shuntaro Takahashi, Yu Chen, “Modeling financial time-series with generative adversarial networks,” Phys. A Stat. Mech. its Appl., vol. 527, p. 121261, 2019, doi: https://doi.org/10.1016/j.physa.2019.121261.

A. Lopez‐Lira, “ Can ChatGPT Forecast Stock Price Movements? ,” Predict. Edge, pp. 121–133, Jul. 2024, doi: 10.1002/9781394308286.CH6.

Chang Che, Zengyi Huang, Chen Li, Haotian Zheng, Xinyu Tian, “Integrating Generative AI into Financial Market Prediction for Improved Decision Making,” arXiv:2404.03523, 2024, [Online]. Available: https://arxiv.org/abs/2404.03523

Dat Mai, “StockGPT: A generative AI model for stock market prediction and trading,” arXiv:2404.05101, 2024, [Online]. Available: https://arxiv.org/abs/2404.05101

Kemal Kirtac, Guido Germano, “Sentiment trading with large language models,” arXiv:2412.19245, 2024, [Online]. Available: https://arxiv.org/abs/2412.19245

Paolo Pedinotti, Peter Baumann, Nathan Jessurun, Leslie Barrett, Enrico Santus, “MetaGraph: A Large-Scale Meta-Analysis of GenAI in Financial NLP (2022-2025),” arXiv:2509.09544, 2026, [Online]. Available: https://arxiv.org/abs/2509.09544

Bruno Miranda Henrique, Vinicius Amorim Sobreiro, Herbert Kimura, “Literature review: Machine learning techniques applied to financial market prediction,” Expert Syst. Appl., vol. 124, pp. 226–251, 2019, [Online]. Available: https://www.sciencedirect.com/science/article/pii/S095741741930017X

F. X. Diebold and R. S. Mariano, “Comparing Predictive Accuracy,” J. Bus. Econ. Stat., vol. 13, no. 3, p. 253, Jul. 1995, doi: 10.2307/1392185.

David Harvey, Stephen Leybourne, “Testing the equality of prediction mean squared errors,” Int. J. Forecast., vol. 13, no. 2, pp. 281–291, 1997.

I. Gufler, F. Sangiorgi, and E. Tarantino, “(Deep) Learning to Trade: An Analysis of AI Trading and Market Outcomes *,” Mar. 2025, doi: 10.2139/SSRN.5375160.