Artificial intelligence research in the stock market sector has been heavily geared towards stock price prediction rather than stock price manipulation. As online trading systems have increased the amount of high volume and re-al-time data transactions, the stock market has increased vulnerability to at-tacks. This paper aims to detect these attacks based on normal trade behavior using an Artificial Immune System (AIS) approach combined with one of four clustering algorithms. The AIS approach is inspired by its proven ability to handle time-series data and its ability to detect abnormal behavior while only being trained on regular trade behavior. These two main points are essential as the models need to adapt over time to adjust to normal trade behavior as it evolves, and due to confidentiality and data restrictions, real-world manipula-tions are not available for training. This paper discovers a competitive alterna-tive to the leading approach and investigates the effects of combining AIS with clustering algorithms; Kernel Density Estimation, Self-Organized Maps, Densi-ty-Based Spatial Clustering of Applications with Noise and Spectral clustering. The best performing solution achieves leading performance using common clustering metrics, including Area Under the Curve, False Alarm Rate, False Negative Rate, and Computation Time.
KeywordsArtificial Immune SystemClusteringAnomaly DetectionFinancial Data
Leangarun, T., Tangamchit, P. and Thajchayapong, S. (2018) Stock Price Manipulation Detection using Generative Adversarial Networks. 2018 IEEE Symposium Series on Computational Intelligence (SSCI), Bangalore, 18-21 November 2018, 2104-2111. https://doi.org/10.1109/SSCI.2018.8628777
Rizvi, B., Belatreche, A. and Bouridane, A. (2019) A Dendritic Cell Immune System Inspired Approach for Stock Market Manipulation Detection. 2019 IEEE Congress on Evolutionary Computation (CEC), Wellington, 10-13 June 2019, 3325-3332. https://doi.org/10.1109/CEC.2019.8789938
Ahmed, M., Choudhury, N. and Uddin, S. (2017) Anomaly Detection on Big Data in Financial Markets. 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), Sydney, July 2017, 998-1001. https://doi.org/10.1145/3110025.3119402
Cao, Y., Li, Y., Coleman, S., Belatreche, A. and McGinnity, T.M. (2014) Detecting Price Manipulation in the Financial Market. 2014 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr), London, 27-28 March 2014, 77-84. https://doi.org/10.1109/CIFEr.2014.6924057
Golmohammadi, K. and Zaiane, O.R. (2015) Time Series Contextual Anomaly Detection for Detecting Market Manipulation in Stock Market. 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA), Paris, 19-21 October 2015, 1-10. https://doi.org/10.1109/DSAA.2015.7344856
Fan, S.S., Liu, G.H. and Chen, Z. (2017) Anomaly Detection Methods for Bankruptcy Prediction. 2017 4th International Conference on Systems and Informatics (ICSAI), Hangzhou, 11-13 November 2017, 1456-1460. https://doi.org/10.1109/ICSAI.2017.8248515
Raza, M. and Qayyum, U. (2019) Classical and Deep Learning Classifiers for Anomaly Detection. 2019 16th International Bhurban Conference on Applied Sciences and Technology (IBCAST), Islamabad, 8-12 January 2019, 614-618. https://doi.org/10.1109/IBCAST.2019.8667245
Samarakoon, P.A. and Athukorala, D.A.S. (2017) System Abnormality Detection in Stock Market Complex Trading Systems Using Machine Learning Techniques. 2017 National Information Technology Conference (NITC), Colombo, 14-15 September 2017, 125-130. https://doi.org/10.1109/NITC.2017.8285660
Anandharaj, A. and Sivakumar, P.B. (2019) Anomaly Detection in Time Series Data Using Hierarchical Temporal Memory Model. 2019 3rd International Conference on Electronics, Communication and Aerospace Technology (ICECA), Coimbatore, 12-14 June 2019, 1287-1292. https://doi.org/10.1109/ICECA.2019.8821966
Cao, Y., Li, Y., Coleman, S., Belatreche, A. and McGinnity, T.M. (2015) Adaptive Hidden Markov Model with Anomaly States for Price Manipulation Detection. IEEE Transactions on Neural Networks and Learning Systems, 26, 318-330. https://doi.org/10.1109/TNNLS.2014.2315042
Liu, J.M., Tian, J., Cai, Z.X., Zhou, Y., Luo, R.H. and Wang, R.R. (2017) A Hybrid Semi-Supervised Approach for Financial Fraud Detection. 2017 International Conference on Machine Learning and Cybernetics (ICMLC), Ningbo, 9-12 July 2017, 217-222. https://doi.org/10.1109/ICMLC.2017.8107767
Li, M., Kashef, R. and Ibrahim, A. (2020) Multi-Level Clustering-Based Outlier’s Detection (MCOD) Using Self-Organizing Maps. Big Data and Cognitive Computing, 4, 24. https://doi.org/10.3390/bdcc4040024
Al-Thani, H., Hassen, H., Al-Maadced, S., Fetais, N. and Jaoua, A. (2018) Unsupervised Technique for Anomaly Detection in Qatar Stock Market. 2018 International Conference on Computer and Applications (ICCA), Beirut, 25-26 August 2018, 116-119. https://doi.org/10.1109/COMAPP.2018.8460282
Camino, R.D., State, R., Montero, L. and Valtchev, P. (2017) Finding Suspicious Activities in Financial Transactions and Distributed Ledgers. 2017 IEEE International Conference on Data Mining Workshops (ICDMW), New Orleans, 18-21 November 2017, 787-796. https://doi.org/10.1109/ICDMW.2017.109
Alizadeh, E., Meskin, N. and Khorasani, K. (2018) A Dendritic Cell Immune System Inspired Scheme for Sensor Fault Detection and Isolation of Wind Turbines. IEEE Transactions on Industrial Informatics, 14, 545-555. https://doi.org/10.1109/TII.2017.2746761
Mokhtar, M., Bi, R., Timmis, J. and Tyrrell, A.M. (2009) A modified Dendritic Cell Algorithm for On-Line Error Detection in Robotic Systems. 2009 IEEE Congress on Evolutionary Computation, Trondheim, 18-21 May 2009, 2055-2062. https://doi.org/10.1109/CEC.2009.4983194
Wang, L. and Fang, X.J. (2015) The Detection of P2P Bots Using the Dendritic Cells Algorithm. 2015 International Conference on Estimation, Detection and Information Fusion (ICEDIF), Harbin, 10-11 January 2015, 299-302. https://doi.org/10.1109/ICEDIF.2015.7280211
Huang, Y., Chung, W. and Tang, X. (2018) A Temporal Recurrent Neural Network Approach to Detecting Market Anomaly Attacks. 2018 IEEE International Conference on Intelligence and Security Informatics (ISI), Miami, 9-11 November 2018, 160-162. https://doi.org/10.1109/ISI.2018.8587397
De Mello Honorio, L., Da Silva, A.M.L. and Barbosa, D.A. (2012) A Cluster and Gradient-Based Artificial Immune System Applied in Optimization Scenarios. IEEE Transactions on Evolutionary Computation, 16, 301-318. https://doi.org/10.1109/TEVC.2010.2044242
Freitas, A.A. and Timmis, J. (2007) Revisiting the Foundations of Artificial Immune Systems for Data Mining. IEEE Transactions on Evolutionary Computation, 11, 521-540. https://doi.org/10.1109/TEVC.2006.884042
Kashef, R.F. (2018) Proceedings of the KDD 2017: Workshop on Anomaly Detection in Finance, PMLR 71. 43-55.
Deng, Y., Bao, F., Kong, Y., Ren, Z. and Dai, Q. (2017) Deep Direct Reinforcement Learning for Financial Signal Representation and Trading. IEEE Transactions on Neural Networks and Learning Systems, 28, 653-664. https://doi.org/10.1109/TNNLS.2016.2522401
Leangarun, T., Tangamchit, P. and Thajchayapong, S. (2016) Stock Price Manipulation Detection Using a Computational Neural Network Model. 2016 Eighth International Conference on Advanced Computational Intelligence (ICACI), Chiang Mai, 14-16 February 2016, 337-341. https://doi.org/10.1109/ICACI.2016.7449848
Hu, W.M., Gao, J., Li, B., Wu, O., Du, J.P. and Maybank, S. (2020) Anomaly Detection Using Local Kernel Density Estimation and Context-Based Regression. IEEE Transactions on Knowledge and Data Engineering, 32, 218-233. https://doi.org/10.1109/TKDE.2018.2882404
Ahmed, T. (2009) Online Anomaly Detection Using KDE. GLOBECOM 2009—2009 IEEE Global Telecommunications Conference, Honolulu, 30 November-4 December 2009, 1-8, https://doi.org/10.1109/GLOCOM.2009.5425504
Aquize, V.G., Emery, E. and De Lima Neto, F.B. (2017) Self-Organizing Maps for Anomaly Detection in Fuel Consumption. Case Study: Illegal Fuel Storage in Bolivia. 2017 IEEE Latin American Conference on Computational Intelligence (LA-CCI), Arequipa, 8-10 November 2017, 1-6. https://doi.org/10.1109/LA-CCI.2017.8285697
Sukkhawatchani, P. and Usaha, W. (2008) Performance Evaluation of Anomaly Detection in Cellular Core Networks Using Self-Organizing Map. 2008 5th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology, Krabi, 14-17 May 2008, 361-364. https://doi.org/10.1109/ECTICON.2008.4600446
Ben Salem, M., Ettabaa, K.S. and Bouhlel, M.S. (2016) Anomaly Detection in Hyperspectral Images Based Spatial Spectral Classification. 2016 7th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT), Hammamet, 18-20 December 2016, 166-170. https://doi.org/10.1109/SETIT.2016.7939860
Proft, J., Suarez, J. and Murphy, R. (2015) Spectral Anomaly Detection with Machine Learning for Wilderness Search and Rescue. 2015 IEEE MIT Undergraduate Research Technology Conference (URTC), Cambridge, 7-8 November 2015, 1-3. https://doi.org/10.1109/URTC.2015.7563746
Ranjith, R., Athanesious, J.J. and Vaidehi, V. (2015) Anomaly Detection Using DBSCAN Clustering Technique for Traffic Video Surveillance. 2015 7th International Conference on Advanced Computing (ICoAC), Chennai, 15-17 December 2015, 1-6. https://doi.org/10.1109/ICoAC.2015.7562795
Loi, N.V., Trung Kien, T., Hop, T.V., Thanh Son, L. andKhuong, N.V. (2020) Abnormal Moving Speed Detection Using Combination of Kernel Density Estimator and DBSCAN for Coastal Surveillance Radars. 2020 7th International Conference on Signal Processing and Integrated Networks (SPIN), Noida, 27-28 February 2020, 143-147. https://doi.org/10.1109/SPIN48934.2020.9070885
Elisa, N., Yang, L.Z. and Naik, N. (2018) Dendritic Cell Algorithm with Optimised Parameters Using Genetic Algorithm. 2018 IEEE Congress on Evolutionary Computation (CEC), Rio de Janeiro, 8-13 July 2018, 1-8. https://doi.org/10.1109/CEC.2018.8477932
Ullah, I., Hussain, H., Ali, I. and Liaquat, A. (2019) Churn Prediction in Banking System Using K-Means, LOF, and CBLOF. 2019 International Conference on Electrical, Communication, and Computer Engineering (ICECCE), Swat, 24-25 July 2019, 1-6. https://doi.org/10.1109/ICECCE47252.2019.8940667
Zhao, Y., Nasrullah, Z. and Li, Z. (2019) PyOD: A Python Toolbox for Scalable Outlier Detection. Journal of Machine Learning Research, 20, 1-7.
Tian, J., Azarian, M.H. and Pecht, M. (2014) Anomaly Detection Using Self-Organizing Maps-Based K-Nearest Neighbor Algorithm. European Conference of the Prognostics and Health Management Society.
Das, S., Moore, T., Wong, W.-K., Stumpf, S., Oberst, I., McIntosh, K. and Burnett, M. (2013) End-User Feature Labeling: Supervised and Semi-Supervised Approaches Based on Locally-Weighted Logistic Regression. Artificial Intelligence, 204, 56-74. https://doi.org/10.1016/j.artint.2013.08.003
Das, S. (2018) Ad_examples/ad/spectral_outler.py. Github Repository. https://github.com/shubhomoydas/ad_examples/blob/master/ad_examples/ad/spectral_outlier.py
Kashef, R., Gencarelli, M. and Ibrahim, A. (2020) Classification of Outlier’s Detection Methods Based on Quantitative or Semantic Learning. In: Fadlullah, Z. and Khan Pathan, A.S., Eds., Combating Security Challenges in the Age of Big Data. Advanced Sciences and Technologies for Security Applications, Springer, Cham, 45-59. https://doi.org/10.1007/978-3-030-35642-2_3
Pedregosa, et al. (2011) Scikit-Learn: Machine Learning in Python. Journal of Machine Learning Research, 12, 2825-2830. https://scikit-learn.org/stable/modules/generated/sklearn.cluster.cluster_optics_dbscan.html
Polygon.io. (2020) Polygon. Io-Real-Time Stock Apis, Forex and Crypto. https://polygon.io/
Dautov, C.P. and Ozerdem, M.S. (2018) Wavelet Transform and Signal Denoising Using Wavelet Method. 2018 26th Signal Processing and Communications Applications Conference (SIU), Izmir, 2-5 May 2018, 1-4. https://doi.org/10.1109/SIU.2018.8404418
Ding, Y. and Selesnick, I.W. (2015) Artifact-Free Wavelet Denoising: Non-Convex Sparse Regularization, Convex Optimization. IEEE Signal Processing Letters, 22, 1364-1368. https://doi.org/10.1109/LSP.2015.2406314