A Geometric View on Inner Transformation between the Variables of a Linear Regression Model
- 1 Shanghai University of International Business and Economics, Shanghai, China
- 2 School of Economics and Business, University of Ljubljana, Ljubljana, Slovenia
Abstract
In the teaching and researching of linear regression analysis, it is interesting and enlightening to explore how the dependent variable vector can be inner-transformed into regression coefficient estimator vector from a visible geometrical view. As an example, the roadmap of such inner transformation is presented based on a simple multiple linear regression model in this work. By applying the matrix algorithms like singular value decomposition (SVD) and Moore-Penrose generalized matrix inverse, the dependent variable vector lands into the right space of the independent variable matrix and is metamorphosed into regression coefficient estimator vector through the three-step of inner transformation. This work explores the geometrical relationship between the dependent variable vector and regression coefficient estimator vector as well as presents a new approach for vector rotating.
- Mandel, J. (1982) Use of the Singular Value Decomposition in Regression Analysis. The American Statistician, 36, 15-24. https://doi.org/10.1080/00031305.1982.10482771
- Mishra, A., Dey, D.K. and Chen, K. (2017) Sequential Co-Sparse Factor Regression. Journal of Computational and Graphical Statistics, 26, 814-826. https://doi.org/10.1080/10618600.2017.1340891
- Nelder, J.A. (1985) An Alternative Interpretation of Singular Value Decomposition in Regression. The American Statistician, 39, 63-65. https://doi.org/10.2307/2683911
- De Schutter, B. and De Moor, B. (1998) The QR Decomposition and the Singular Value Decomposition in the Symmetrized Max-Plus Algebra. SIAM Journal on Matrix Analysis and Applications, 19, 378-406. https://doi.org/10.1137/S0895479896304782
- Kalman, D. (1996) A Singularly Valuable Decomposition: SVD of a Matrix. The College Mathematics Journal, 27, 2-23. https://doi.org/10.1080/07468342.1996.11973744
- Stewart G.W. (1993) On the Early History of the Singular Value Decomposition. SIAM Review, 35, 551-566. https://doi.org/10.1137/1035134
- Stewart, G.W. (1998) Matrix Algorithms: Volume 1: Basic Decompositions. Society for Industrial and Applied Mathematics, Philadelphia. https://doi.org/10.1137/1.9781611971408
- Ben-Isreal, A. and Charnes, A. (1963) Contributions to the Theory of Generalized Inverses. Journal of the Society for Industrial and Applied Mathematics, 11, 667-699. https://doi.org/10.1137/0111051
- Campbell, S.L. and Meyer, C.D. (1991) Generalized Inverse of Linear Transformations. Dover Publications, New York.
- Fill, J.A. and Fishkind, D.E. (2000) The Moore—Penrose Generalized Inverse for Sums of Matrices. SIAM Journal on Matrix Analysis and Applications, 21, 629-635. https://doi.org/10.1137/S0895479897329692
- Penrose, R. (1955) A Generalized Inverse for Matrices. Mathematical Proceedings of the Cambridge Philosophical Society, 51, 406-413. https://doi.org/10.1017/S0305004100030401
- Wang, G.R., Wei, Y.M. and Qiao, S. (2003) Generalized Inverse: Theory and Computations. Science Press, Beijing.
- Tian, Y. and Zhang, J. (2011) Some Equalities for Estimations of Partial Coefficients under a General Linear Regression Model. Statistical Papers, 52, 911-920. https://doi.org/10.1007/s00362-009-0298-5
- Xu, W. and Tang, X. (2009) Ridge Estimate on Moore-Penrose Inverse Matrix of Generalized Linear Regression Model. Journal of Northeast Forestry University, 37, 108-119.