Research ArticleOpen AccessGoogle Scholar indexed
Group Variable Selection via a Combination of <i>L</i><sub>q</sub> Norm and Correlation-Based Penalty
Department of Mathematics, College of Science, Shanghai University, Shanghai, China
Department of Mathematics, College of Science, Shanghai University, Shanghai, China
- 1 Department of Mathematics, College of Science, Shanghai University, Shanghai, China
- 2 Department of Mathematics, College of Science, Shanghai University, Shanghai, China
Advances in Pure Mathematics·Volume 07 (2017)·Pages 51–65·Published 23 January 2017·DOI10.4236/apm.2017.71005
Copy link · social · email
Abstract
Considering the problem of feature selection in linear regression model, a new method called LqCP is proposed simultaneously to select variables and favor a grouping effect, where strongly correlated predictors tend to be in or out of the model together. LqCP is based on penalized least squares with a penalty function that combines the L q (0 n. In addition, a simulation about grouped variable selection is performed. Finally, The model is applied to two real data: US Crime Data and Gasoline Data. In terms of prediction error and estimation error, empirical studies show the efficiency of LqCP.
KeywordsLinear RegressionVariable SelectionElastic NetAdaptive Elastic Net
- Fan, J. and Li, R. (2006) Statistical Challenges with High Dimensionality: Feature Selection in Knowledge Discovery. Marta Sanz Solé, 595-622.
- Breiman, L. (1996) Heuristics of Instability and Stabilization in Model Selection. Annals of Statistics, 24, 2350-2383. https://doi.org/10.1214/aos/1032181158
- Tibshirani, R. (1996) Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society: Series B, 58, 267-288.
- Knight, K. and Fu, W.J. (2000) Asymptotics for Lasso-Type Estimators. Annals of Statistics, 28, 1356-1378. https://doi.org/10.1214/aos/1015957397
- Fan, J. and Li, R. (2001) Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties. Journal of the American Statistical Association, 96, 1348-1360. https://doi.org/10.1198/016214501753382273
- Zou, H. (2006) The Adaptive Lasso and Its Oracle Properties. Journal of the American Statistical Association, 101, 1418-1429. https://doi.org/10.1198/016214506000000735
- Zou, H. and Hastie, T. (2005) Regularization and Variable Selection via the Elastic Net. Journal of the Royal Statistical Society: Series B, 67, 301-320. https://doi.org/10.1111/j.1467-9868.2005.00503.x
- Zou, H. and Zhang, H.H. (2009) On the Adaptive Elastic Net with a Diverging Number of Parameters. Annals of Statistics, 37, 1733-1751. https://doi.org/10.1214/08-AOS625
- Ghosh, S. (2011) On the Grouped Selection and Model Complexity of the Adaptive Elastic Net. Statistics and Computing, 21, 451-462. https://doi.org/10.1007/s11222-010-9181-4
- Bondell, H.D. and Reich, B.J. (2008) Simultaneous Regression Shrinkage, Variable Selection and Clustering of Predictors with OSCAR. Biometrics, 64, 115-123. https://doi.org/10.1111/j.1541-0420.2007.00843.x
- EL Anari, M. and Mkhadri, A. (2014) Penalized Regression Combining the L 1 Norm and a Correlation Based Penalty. Sankhya B, 76, 82-102. https://doi.org/10.1007/s13571-013-0065-4
- Tutz, G. and Ulbricht, J. (2009) Penalized Regression with Correlation Based Penalty. Statistics and Computing, 19, 239-253. https://doi.org/10.1007/s11222-008-9088-5
- Frank, I.E. and Friedman, J.H. (1993) An Statistical View of Some Chemometrics Regression Tools. Technometrics, 35, 109-135. https://doi.org/10.1080/00401706.1993.10485033
- Fu. W.J. (1998) Penalized Regressions: The Bridge vs the Lasso. Journal of Computational and Graphical Statistics, 7, 397-416.