A New Extended BIC and Sequential Lasso Regression Analysis and Their Application in Classification

Abstract

In this paper, firstly, we propose a new method for choosing regularization parameter λ for lasso regression, which differs from traditional method such as multifold cross-validation, our new method gives the maximum value of parameter λ directly. Secondly, by considering another prior form over model space in the Bayes approach, we propose a new extended Bayes information criterion family, and under some mild condition, our new EBIC (NEBIC) is shown to be consistent. Then we apply our new method to choose parameter for sequential lasso regression which selects features by sequentially solving partially penalized least squares problems where the features selected in earlier steps are not penalized in the subsequent steps. Then sequential lasso uses NEBIC as the stopping rule. Finally, we apply our algorithm to identify the nonzero entries of precision matrix for high-dimensional linear discrimination analysis. Simulation results demonstrate that our algorithm has a lower misclassification rate and less computation time than its competing methods under considerations.