Research ArticleOpen AccessGoogle Scholar indexed
Choosing Appropriate Regression Model in the Presence of Multicolinearity
Department of Engineering and Mathematics, Sheffield Hallam University, Sheffield, UK
Department of Mathematics & Statistics, University of Uyo, Uyo, Nigeria
Department of Computer Science, Sheffield Hallam University, Sheffield, UK
- 1 Department of Engineering and Mathematics, Sheffield Hallam University, Sheffield, UK
- 2 Department of Mathematics & Statistics, University of Uyo, Uyo, Nigeria
- 3 Department of Computer Science, Sheffield Hallam University, Sheffield, UK
Open Journal of Statistics·Volume 09 (2019)·Pages 159–168·Published 29 March 2019·DOI10.4236/ojs.2019.92012
Copy link · social · email
Abstract
This work is geared towards detecting and solving the problem of multicolinearity in regression analysis. As such, Variance Inflation Factor (VIF) and the Condition Index (CI) were used as measures of such detection. Ridge Regression (RR) and the Principal Component Regression (PCR) were the two other approaches used in modeling apart from the conventional simple linear regression. For the purpose of comparing the two methods, simulated data were used. Our task is to ascertain the effectiveness of each of the methods based on their respective mean square errors. From the result, we found that Ridge Regression (RR) method is better than principal component regression when multicollinearity exists among the predictors.
KeywordsMulticollinearityAdequacyRegression coefficientsVariance Inflation Factor (VIF)Mean Square Error
- Mansfiled, E.R. and Billy, P.H. (1982) Detecting Multicollinearity. The American Statisticians, 36, 158-160. https://doi.org/10.2307/2683167
- Hwang, J.T. and Netleton, D. (2000) Principal Component Regression with Data for Chosen Components and Related Methods. Technometrics, 45, 70-79. https://doi.org/10.1198/004017002188618716
- Montgomery, D.C., Peck, E.A. and Vining, G.G. (2001) Introduction to Linear Regression Analysis. 3rd Edition, Willey, New York.
- Farrar, D.E. and Glanber, R.R. (1967) Multicollinearity in Regression Analysis. Review of Economic and Statistic, 49, 92-107.
- Myers, R.H. (1986) Classical and Modern Regression with Applications. 2nd Edition, PWSKENT Publishing Company, USA.
- Hair, J.F., Anderson, R.E. and Black, W.C. (1998) Multivariate Data Analysis. 5th Edition, Prentice Hall, New Jersey.
- Filzmoser, P. and Groux, C. (2002) A Projection Algorithm for Regression with Collinearity. In: Jajuga, K., Sokolowski, A. and Bock, H.H., Eds., Classification, Clustering, and Data Analysis. Studies in Classification, Data Analysis, and Knowledge Organization, Springer, Berlin, Heidelberg, 227-234.
- Martens, H. and Naes, T. (1989) Assessment, Validation and Choice of Calibration Method. In: Multivariate Calibration, 1st Edition, John Wiley & Sons, New York, 237-266.
- Hoerl, A.E. and Kennard, R.W. (1970). Ridge Regression: Biased Estimation for Non-Orthogonal Problems. Technometrics, 12, 55-67. https://doi.org/10.1080/00401706.1970.10488634
- Bjokstrom, A. (2001) Ridge Regression and Inverse Problem. Research Report, Stockholm University, Stockholm.
- Marquardt, D.W. (1963) An Algorithm for Least-Squares Estimation of Nonlinear Parameters. Journal of Society of Industrial Applied Mathematics, 11, 431-441.
- Marquardt, D.W. (1970) Generalized Inverses, Ridge Regression. Biased Linear Estimation, and Nonlinear Estimation, Techno Metrics, 12, 591-612. https://doi.org/10.2307/1267205
- Ijomah, M.A. and Nwakuya M.T. (2011) Ridge Regression Estimate. Unpublished M.Sc. Thesis, University of Port Harcourt, Port Harcourt.
- Pasha, G.R. and Shah, M.A. (2004) Application of Ridge Regression to Multicollinearity Data. Journal of Research (Science), 15, 97-106.