Research ArticleOpen AccessGoogle Scholar indexed

LP-SVR Model Selection Using an Inexact Globalized Quasi-Newton Strategy

¹ Department of Computer Science, Baylor University, Waco, TX, USA
² Department of Electrical and Computer Engineering, Auto- nomous University of Ciudad Juarez, Ciudad Juárez, Mexico
³ Department of Electrical and Computer Engineering, Auto- nomous University of Ciudad Juarez, Ciudad Juárez, Mexico
⁴ Department of Electrical and Computer Engineering, Auto- nomous University of Ciudad Juarez, Ciudad Juárez, Mexico
⁵ Science Applications International Corporation, El Paso, TX, USA.

Journal of Intelligent Learning Systems and Applications·Volume 05 (2013)·Pages 19–28·Published 22 February 2013·DOI10.4236/jilsa.2013.51003

Abstract

In this paper we study the problem of model selection for a linear programming-based support vector machine for re gression. We propose generalized method that is based on a quasi-Newton method that uses a globalization strategy and an inexact computation of first order information. We explore the case of two-class, multi-class, and regression prob lems. Simulation results among standard datasets suggest that the algorithm achieves insignificant variability when measuring residual statistical properties.

KeywordsHyper-Parameter EstimationSupport Vector RegressionMachine LearningData Mining

A. J. Smola and B. Scholkopf, “A Tutorial on Support Vector Regression,” Statistics and Computing, Vol. 14, No. 3, 2004, pp. 199-222. doi:10.1023/B:STCO.0000035301.49549.88
D. Anguita, A. Boni, S. Ridella, F. Rivieccio and D. Sterpi, “Theoretical and Practical Model Selection Methods for Support Vector Classifiers,” Support Vector Machines: Theory and Applications, Vol. 177, 2005, pp. 159-179. doi:10.1007/10984697_7
K. Duan, S. Keerthi and A. Poo, “Evaluation of Simple Performance Measures for Tuning SVM Hyperparameters,” Neurocomputing, Vol. 51, 2003, pp. 41-59. doi:10.1016/S0925-2312(02)00601-X
Z. Hui-ren and P. Zheng, “Method for Selecting Parameters of Least Squares Support Vector Machines Based on GA and Bootstrap,” Journal of System Simulation, Vol. 12, 2008.
D. Anguita, S. Ridella, F. Rivieccio and R. Zunino, “Hyperparameter Design Criteria for Support Vector Classifiers,” Neurocomputing, Vol. 55, No. 1-2, 2003, pp. 109-134. doi:10.1016/S0925-2312(03)00430-2
L. Wang and S. O. Service, “Support Vector Machines: Theory and Applications,” Studies in Fuzziness and Soft Computing, Springer-Verlag, Berlin, 2005.
G. Cawley, “Leave-One-Out Cross-Validation Based Model Selection Criteria for Weighted Ls-Svms,” IEEE International Conference on Neural Networks, 16-21 July 2006. doi:10.1109/IJCNN.2006.246634
P. R. Perea, “Algorithms for Training Large-Scale Linear Programming Support Vector Regression and Classification,” Ph.D. Thesis, The University of Texas, El Paso, 2011.
J. Dennis and R. Schnabel, “Numerical Methods for Unconstrained Optimization and Nonlinear Equations,” Society for Industrial Mathematics, 1996. doi:10.1137/1.9781611971200
M. Argaez and L. Velazquez, “A New Infeasible InteriorPoint Algorithm for Linear Programming,” Proceedings of the 2003 Conference on Diversity in Computing, ACM, New York, 2003, pp. 12-14. http://doi.acm.org/10.1145/948542.948545
J. Mercer, “Functions of Positive and Negative Type, and Their Connection with the Theory of Integral Equations,” Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, Vol. 209, No. 441-458, 1909, pp. 415-446. doi:10.1098/rsta.1909.0016
R. Courant and D. Hilbert, “Methods of Mathematical Physics,” Interscience, New York, 1966.
Z. Lu, J. Sun and K. R. Butts, “Linear Programming Support Vector Regression with Wavelet Kernel: A New Approach to Nonlinear Dynamical Systems Identification,” Mathematics and Computers in Simulation, Vol. 79, No. 7, 2009, pp. 2051-2063. doi:10.1016/j.matcom.2008.10.011

Menu

LP-SVR Model Selection Using an Inexact Globalized Quasi-Newton Strategy

Abstract

References (18)