Main Article Content
Abstract
The increasing complexity and volatility of modern financial markets necessitate advanced anomaly detection systems that can identify irregular patterns, which may signal market manipulation, systemic risks, or emerging crises. This research presents a comprehensive deep learning framework for real-time anomaly detection in stock markets, integrated with business decision support systems to enhance risk management and regulatory compliance. We propose and evaluate four distinct deep learning architectures: LSTM-Autoencoder, Variational Autoencoder (VAE), Transformer-based models, and an ensemble approach, utilizing high-frequency trading data from major stock exchanges spanning 2019-2024. Our methodology incorporates multi-dimensional feature engineering, including technical indicators, market microstructure variables, and sentiment analysis, processed through advanced normalization techniques. The experimental results demonstrate that the Transformer-based ensemble model achieves superior performance with an F1-score of 0.89 and AUC of 0.94, representing a 43.5% improvement over traditional methods (F1=0.62 for ARIMA-GARCH) and 17% improvement over standalone machine learning approaches (F1=0.76 for XGBoost). The system successfully detected 92% of major market anomalies with a 15-minute average early warning time while maintaining a false positive rate below 3%. Furthermore, the integration with decision support systems yielded a 34% improvement in risk-adjusted returns for test portfolios, reducing decision-making time by 67.3% (from 98s to 32s) and achieving cost savings of $35.2M monthly across deployed institutions. This research contributes to financial technology by bridging the gap between advanced deep learning techniques and practical business applications, offering a scalable solution for market surveillance and risk management in increasingly complex financial ecosystems.
Keywords
Article Details
References
- H. Wu, J. Xu, J. Wang, M. Long, Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting, Advances in neural information processing systems 34 (2021) 22419-22430.
- https://doi.org/10.48550/arXiv.2106.13008
- Z. Zamanzadeh Darban, G.I. Webb, S. Pan, C. Aggarwal, M. Salehi, Deep learning for time series anomaly detection: A survey, ACM Computing Surveys 57(1) (2024) 1-42.
- https://doi.org/10.1145/3691338
- G. Zissis, P. Bertoldi, Update on status of solid-state lighting & smart lighting systems, 2023. https://doi.org/10.2760/223640
- S. Tuli, G. Casale, N.R. Jennings, Tranad: Deep transformer networks for anomaly detection in multivariate time series data, arXiv preprint arXiv:2201.07284 (2022).https://doi.org/10.48550/arXiv.2201.07284
- K. Biriukova, A. Bhattacherjee, Using transformer models for stock market anomaly detection, Journal of Data Science 2023(21) (2023) 1-8. https://doi.org/10.61453/jods.v2023no21
- H.D. Nguyen, K.P. Tran, S. Thomassey, M. Hamad, Forecasting and Anomaly Detection approaches using LSTM and LSTM Autoencoder techniques with the applications in supply chain management, International Journal of Information Management 57 (2021) 102282. https://doi.org/10.1016/j.ijinfomgt.2020.102282
- Y. Wei, J. Jang-Jaccard, W. Xu, F. Sabrina, S. Camtepe, M. Boulic, LSTM-autoencoder-based anomaly detection for indoor air quality time-series data, IEEE Sensors Journal 23(4) (2023) 3787-3800. https://doi.org/10.1109/JSEN.2022.3230361
- X. Wang, D. Pi, X. Zhang, H. Liu, C. Guo, Variational transformer-based anomaly detection approach for multivariate time series, Measurement 191 (2022) 110791. https://doi.org/10.1016/j.measurement.2022.110791
- C. Wang, Y. Chen, S. Zhang, Q. Zhang, Stock market index prediction using deep Transformer model, Expert Systems with Applications 208 (2022) 118128. https://doi.org/10.1016/j.eswa.2022.118128
- S. Li, X. Huang, Z. Cheng, W. Zou, Y. Yi, AE-ACG: A novel deep learning-based method for stock price movement prediction, Finance Research Letters 58 (2023) 104304. https://doi.org/10.1016/j.frl.2023.104304
- R. Bhatt, A. Kumari, S.B. Rajasekaran, V.P. Deshmukh, A. Srivastava, Technique for forecasting future market movement using machine learning and deep learning algorithms, 2023 3rd international conference on advance computing and innovative technologies in engineering (ICACITE), IEEE, 2023, pp. 471-474. https://doi.org/10.1109/ICACITE57410.2023.10183197
- S. Kumari, C. Prabha, A. Karim, M.M. Hassan, S. Azam, A Comprehensive Investigation of Anomaly Detection Methods in Deep Learning and Machine Learning: 2019–2023, IET Information Security 2024(1) (2024) 8821891. https://doi.org/10.1049/2024/8821891
- Y. Xiang, Using ARIMA‐GARCH Model to Analyze Fluctuation Law of International Oil Price, Mathematical Problems in Engineering 2022(1) (2022) 3936414. https://doi.org/10.1155/2022/3936414
- Q. Zhang, Financial data anomaly detection method based on decision tree and random forest algorithm, Journal of Mathematics 2022(1) (2022) 9135117. https://doi.org/10.1155/2022/9135117
- W. Hilal, S.A. Gadsden, J. Yawney, Financial fraud: a review of anomaly detection techniques and recent advances, Expert systems With applications 193 (2022) 116429. https://doi.org/10.1016/j.eswa.2021.116429
- C. Zhang, N.N.A. Sjarif, R. Ibrahim, Deep learning models for price forecasting of financial time series: A review of recent advancements: 2020–2022, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 14(1) (2024) e1519. https://doi.org/10.1002/widm.1519
- G. Fatouros, G. Makridis, D. Kotios, J. Soldatos, M. Filippakis, D. Kyriazis, DeepVaR: a framework for portfolio risk assessment leveraging probabilistic deep neural networks, Digital finance 5(1) (2023) 29-56. https://doi.org/10.1007/s42521-022-00050-0
- S. Gu, B. Kelly, D. Xiu, Empirical asset pricing via machine learning, The Review of Financial Studies 33(5) (2020) 2223-2273. https://doi.org/10.1093/rfs/hhaa009
- F. Lachekhab, M. Benzaoui, S.A. Tadjer, A. Bensmaine, H. Hamma, LSTM-autoencoder deep learning model for anomaly detection in electric motor, Energies 17(10) (2024) 2340. https://doi.org/10.3390/en17102340
- L. Xu, K. Xu, Y. Qin, Y. Li, X. Huang, Z. Lin, N. Ye, X. Ji, TGAN-AD: Transformer-based GAN for anomaly detection of time series data, Applied Sciences 12(16) (2022) 8085. https://doi.org/10.3390/app12168085
- K. Choi, J. Yi, C. Park, S. Yoon, Deep learning for anomaly detection in time-series data: Review, analysis, and guidelines, IEEE access 9 (2021) 120043-120065. https://doi.org/10.1109/ACCESS.2021.3107975
- M. Soori, F.K.G. Jough, R. Dastres, B. Arezoo, AI-based decision support systems in Industry 4.0, A review, Journal of Economy and Technology (2024). https://doi.org/10.1016/j.ject.2024.08.005
- M. Schmitt, Automated machine learning: AI-driven decision making in business analytics, Intelligent Systems with Applications 18 (2023) 200188. https://doi.org/10.1016/j.iswa.2023.200188
- L. Wang, Z. Zhang, D. Wang, W. Cao, X. Zhou, P. Zhang, J. Liu, X. Fan, F. Tian, Human-centered design and evaluation of AI-empowered clinical decision support systems: a systematic review, Frontiers in Computer Science 5 (2023) 1187299. https://doi.org/10.3389/fcomp.2023.1187299
References
H. Wu, J. Xu, J. Wang, M. Long, Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting, Advances in neural information processing systems 34 (2021) 22419-22430.
https://doi.org/10.48550/arXiv.2106.13008
Z. Zamanzadeh Darban, G.I. Webb, S. Pan, C. Aggarwal, M. Salehi, Deep learning for time series anomaly detection: A survey, ACM Computing Surveys 57(1) (2024) 1-42.
https://doi.org/10.1145/3691338
G. Zissis, P. Bertoldi, Update on status of solid-state lighting & smart lighting systems, 2023. https://doi.org/10.2760/223640
S. Tuli, G. Casale, N.R. Jennings, Tranad: Deep transformer networks for anomaly detection in multivariate time series data, arXiv preprint arXiv:2201.07284 (2022).https://doi.org/10.48550/arXiv.2201.07284
K. Biriukova, A. Bhattacherjee, Using transformer models for stock market anomaly detection, Journal of Data Science 2023(21) (2023) 1-8. https://doi.org/10.61453/jods.v2023no21
H.D. Nguyen, K.P. Tran, S. Thomassey, M. Hamad, Forecasting and Anomaly Detection approaches using LSTM and LSTM Autoencoder techniques with the applications in supply chain management, International Journal of Information Management 57 (2021) 102282. https://doi.org/10.1016/j.ijinfomgt.2020.102282
Y. Wei, J. Jang-Jaccard, W. Xu, F. Sabrina, S. Camtepe, M. Boulic, LSTM-autoencoder-based anomaly detection for indoor air quality time-series data, IEEE Sensors Journal 23(4) (2023) 3787-3800. https://doi.org/10.1109/JSEN.2022.3230361
X. Wang, D. Pi, X. Zhang, H. Liu, C. Guo, Variational transformer-based anomaly detection approach for multivariate time series, Measurement 191 (2022) 110791. https://doi.org/10.1016/j.measurement.2022.110791
C. Wang, Y. Chen, S. Zhang, Q. Zhang, Stock market index prediction using deep Transformer model, Expert Systems with Applications 208 (2022) 118128. https://doi.org/10.1016/j.eswa.2022.118128
S. Li, X. Huang, Z. Cheng, W. Zou, Y. Yi, AE-ACG: A novel deep learning-based method for stock price movement prediction, Finance Research Letters 58 (2023) 104304. https://doi.org/10.1016/j.frl.2023.104304
R. Bhatt, A. Kumari, S.B. Rajasekaran, V.P. Deshmukh, A. Srivastava, Technique for forecasting future market movement using machine learning and deep learning algorithms, 2023 3rd international conference on advance computing and innovative technologies in engineering (ICACITE), IEEE, 2023, pp. 471-474. https://doi.org/10.1109/ICACITE57410.2023.10183197
S. Kumari, C. Prabha, A. Karim, M.M. Hassan, S. Azam, A Comprehensive Investigation of Anomaly Detection Methods in Deep Learning and Machine Learning: 2019–2023, IET Information Security 2024(1) (2024) 8821891. https://doi.org/10.1049/2024/8821891
Y. Xiang, Using ARIMA‐GARCH Model to Analyze Fluctuation Law of International Oil Price, Mathematical Problems in Engineering 2022(1) (2022) 3936414. https://doi.org/10.1155/2022/3936414
Q. Zhang, Financial data anomaly detection method based on decision tree and random forest algorithm, Journal of Mathematics 2022(1) (2022) 9135117. https://doi.org/10.1155/2022/9135117
W. Hilal, S.A. Gadsden, J. Yawney, Financial fraud: a review of anomaly detection techniques and recent advances, Expert systems With applications 193 (2022) 116429. https://doi.org/10.1016/j.eswa.2021.116429
C. Zhang, N.N.A. Sjarif, R. Ibrahim, Deep learning models for price forecasting of financial time series: A review of recent advancements: 2020–2022, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 14(1) (2024) e1519. https://doi.org/10.1002/widm.1519
G. Fatouros, G. Makridis, D. Kotios, J. Soldatos, M. Filippakis, D. Kyriazis, DeepVaR: a framework for portfolio risk assessment leveraging probabilistic deep neural networks, Digital finance 5(1) (2023) 29-56. https://doi.org/10.1007/s42521-022-00050-0
S. Gu, B. Kelly, D. Xiu, Empirical asset pricing via machine learning, The Review of Financial Studies 33(5) (2020) 2223-2273. https://doi.org/10.1093/rfs/hhaa009
F. Lachekhab, M. Benzaoui, S.A. Tadjer, A. Bensmaine, H. Hamma, LSTM-autoencoder deep learning model for anomaly detection in electric motor, Energies 17(10) (2024) 2340. https://doi.org/10.3390/en17102340
L. Xu, K. Xu, Y. Qin, Y. Li, X. Huang, Z. Lin, N. Ye, X. Ji, TGAN-AD: Transformer-based GAN for anomaly detection of time series data, Applied Sciences 12(16) (2022) 8085. https://doi.org/10.3390/app12168085
K. Choi, J. Yi, C. Park, S. Yoon, Deep learning for anomaly detection in time-series data: Review, analysis, and guidelines, IEEE access 9 (2021) 120043-120065. https://doi.org/10.1109/ACCESS.2021.3107975
M. Soori, F.K.G. Jough, R. Dastres, B. Arezoo, AI-based decision support systems in Industry 4.0, A review, Journal of Economy and Technology (2024). https://doi.org/10.1016/j.ject.2024.08.005
M. Schmitt, Automated machine learning: AI-driven decision making in business analytics, Intelligent Systems with Applications 18 (2023) 200188. https://doi.org/10.1016/j.iswa.2023.200188
L. Wang, Z. Zhang, D. Wang, W. Cao, X. Zhou, P. Zhang, J. Liu, X. Fan, F. Tian, Human-centered design and evaluation of AI-empowered clinical decision support systems: a systematic review, Frontiers in Computer Science 5 (2023) 1187299. https://doi.org/10.3389/fcomp.2023.1187299