Research ArticleOpen AccessGoogle Scholar indexed
An Efficient Smooth Quantile Boost Algorithm for Binary Classification
Department of Mathematics, College of Science, Shanghai University, Shanghai, China
Department of Mathematics, College of Science, Shanghai University, Shanghai, China
- 1 Department of Mathematics, College of Science, Shanghai University, Shanghai, China
- 2 Department of Mathematics, College of Science, Shanghai University, Shanghai, China
Advances in Pure Mathematics·Volume 06 (2016)·Pages 615–624·Published 2 August 2016·DOI10.4236/apm.2016.69050
Copy link · social · email
Abstract
In this paper, we propose a Smooth Quantile Boost Classification (SQBC) algorithm for binary classification problem. The SQBC algorithm directly uses a smooth function to approximate the “check function” of the quantile regression. Compared to other boost-based classification algorithms, the proposed algorithm is more accurate, flexible and robust to noisy predictors. Furthermore, the SQBC algorithm also can work well in high dimensional space. Extensive numerical experiments show that our proposed method has better performance on randomly simulations and real data.
KeywordsBoostingQuantile RegressionSmooth Check FunctionBinary Classification
- Freund, Y. and Schapire, R. (1997) A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. Journal of Computer and System Sciences, 55, 119-139. http://dx.doi.org/10.1006/jcss.1997.1504
- Friedman, J., Hastie, T. and Tibshirani, R. (2000) Additive Logistic Regression: A Statistical View of Boosting. Annals of Statistics, 28, 337-407. http://dx.doi.org/10.1214/aos/1016218223
- Friedman, J. (2001) Greedy Function Approximation: A Gradient Boosting Machine. Annals of Statistics, 29, 1189-1232. http://dx.doi.org/10.1214/aos/1013203451
- Mason, L., Baxter, J., Bartlett, P.L. and Frean, M. (2000) Boosting Algorithms as Gradient Descent. Proceedings of Advances in Neural Information Processing Systems (NIPS), 8, 479-485.
- Bühlmann, P. and Yu, B. (2003) Boosting with the L2-Loss: Regression and Classification. Journal of the American Statistical Association, 98, 324-340. http://dx.doi.org/10.1198/016214503000125
- Kriegler, B. and Berk, R. (2007) Boosting the Quantile Distribution: A Cost-Sensitive Statistical Learning Procedure. Technical Report, Department of Statitics, University of California, Los Angeles.
- Zheng, S.F. (2010) Boosting Based Conditional Quantile Estimation for Regression and Binary Classification. Advances in Soft Computing, 6438, 67-79. http://dx.doi.org/10.1007/978-3-642-16773-7_6
- Zheng, S.F. (2011) Gradient Descent Algorithms for Quantile Regression with Smooth Approximation. International Journal of Machine Learning and Cybernetics, 2, 191-207. http://dx.doi.org/10.1007/s13042-011-0031-2
- Koenker, R. and Bassett, G. (1978) Regression Quantiles. Econometrica, 46, 33-50. http://dx.doi.org/10.2307/1913643
- Koenker, R. (2005) Quantile Regression. Cambridge University Press, New York. http://dx.doi.org/10.1017/CBO9780511754098
- Hunter, D.R. and Lange, K. (2000) Quantile Regression via an MM Algorithm. Journal of Computational and Graphical Statistics, 9, 60-77.
- Chen, C. and Mangasarian, O.L. (1996) A Class of Smoothing Functions for Nonlinear and Mixed Complementarity Problems. Computational Optimization and Applications, 5, 97-138. http://dx.doi.org/10.1007/BF00249052
- Kordas, G. (2006) Smoothed Binary Regression Quantiles. Journal of Applied Econometrics, 21, 387-407. http://dx.doi.org/10.1002/jae.843
- Dettling, M. and Bühlmann, P. (2003) Boosting for Tumor Classification with Gene Expression Data. Bioinformatics, 19, 1061-1069. http://dx.doi.org/10.1093/bioinformatics/btf867