Adaptive Optimization Swarm Algorithm Ensemble Model Applied to the Classification of Unbalanced Data
- 1 School of Business Administration, Shandong University of Finance and Economics, Jinan, China
- 2 School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China
Abstract
In order to solve the problem that, the hyper-parameters of the existing random forest-based classification prediction model depend on empirical settings, which leads to unsatisfactory model performance. We propose a based on adaptive particle swarm optimization algorithm random forest model to optimize data classification and an adaptive particle swarm algorithm for optimizing hyper-parameters in the random forest to ensure that the model can better predict unbalanced data. Aiming at the premature convergence problem in the particle swarm optimization algorithm, the population is adaptively divided according to the fitness of the population, and an adaptive update strategy is introduced to enhance the ability of particles to jump out of the local optimum. The main steps of the model are as follows: Normalize the data set, initialize the model on the training set, and then use the particle swarm optimization algorithm to optimize the modeling process to establish a classification model. Experimental results show that our proposed algorithm is better than traditional algorithms, especially in terms of F1-Measure and ACC evaluation standards. The results of the six-keel imbalanced data set demonstrate the advantages of our proposed algorithm.
- He, H.L. and Fan, Y.L. (2021) A Novel Hybrid Ensemble Model Based on Tree-Based Method and Deep Learning Method for Default Prediction. Expert Systems with Applications, 176, Article No. 114899. https://doi.org/10.1016/j.eswa.2021.114899
- Zhang, H., Li, J.L., Liu, X.M. and Dong, C. (2021) Multidimensional Feature Fusion and Stacking Ensemble Mechanism for Network Intrusion Detection. Future Generation Computer Systems, 122, 130-143. https://doi.org/10.1016/j.future.2021.03.024
- Abkenar, S.B., Mahdipour, E., Jameii, S.M. and Kashani, M.H. (2021) A Hybrid Classification Method for Twitter Spam Detection Based on Differential Evolution and Random Forest. Concurrency and Computation: Practice & Experience.
- Li, H.X., Feng, A., Lin, B., Su, H.C., Liu, Z.X., Duan, X.L., Pu, H.B. and Wang, Y.F. (2021) A Novel Method for Credit Scoring Based on Feature Transformation and Ensemble Model. PeerJ Computer Science, 7, e579.
- Sun, Y.M., Wong, A.K.C. and Kamel, M.S. (2009) Classification of Imbalanced Data: A Review. International Journal of Pattern Recognition & Artificial Intelligence, 23, 687-719. https://doi.org/10.1142/S0218001409007326
- Janicka, M., Lango, M. and Stefanowski, J. (2019) Using Information on Class Interrelations to Improve Classification of Multiclass Imbalanced Data: A New Resampling Algorithm. International Journal of Applied Mathematics and Computer Science, 29, 769-781. https://doi.org/10.2478/amcs-2019-0057
- Pei, W., Xue, B., Shang, L. and Zhang, M. (2019) Genetic Programming for Development of Cost-Sensitive Classifiers for Binary High-Dimensional Unbalanced Classification. Applied Soft Computing, 101, Article No. 106989. https://doi.org/10.1016/j.asoc.2020.106989
- Wolpert, D.H. and Macready, W.G. (1997) No Free Lunch Theorems for Optimization. IEEE Transactions on Evolutionary Computation, 1, 67-82. https://doi.org/10.1109/4235.585893
- Dietterich, T.G. (2000) Ensemble Methods in Machine Learning. In: MCS 2000: Multiple Classifier Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45014-9_1
- Kearns, M. and Valiant, L.G. (1989) Cryptographic Limitations on Learning Boolean Formulae and Finite Automata. 21st Annual ACM Symposium on the Theory of Computing, Seattle, Washington, USA, May 14-17 1989, 433-444. https://doi.org/10.1145/73007.73049
- Breiman, L. (2001) Random Forest. Machine Learning, 45, 5-32. https://doi.org/10.1023/A:1010933404324