Research ArticleOpen AccessGoogle Scholar indexed
Convergence Analysis of a Kind of Deterministic Discrete-Time PCA Algorithm
Department of Mathematics, College of Science, Shanghai University, Shanghai, China
Department of Mathematics, College of Science, Shanghai University, Shanghai, China
Department of Mathematics, College of Science, Shanghai University, Shanghai, China
- 1 Department of Mathematics, College of Science, Shanghai University, Shanghai, China
- 2 Department of Mathematics, College of Science, Shanghai University, Shanghai, China
- 3 Department of Mathematics, College of Science, Shanghai University, Shanghai, China
Advances in Pure Mathematics·Volume 11 (2021)·Pages 408–426·Published 17 May 2021·DOI10.4236/apm.2021.115028
Copy link · social · email
Abstract
We proposed a generalized adaptive learning rate (GALR) PCA algorithm, which could be guaranteed that the algorithm’s convergence process would not be affected by the selection of the initial value. Using the deterministic discrete time (DDT) method, we gave the upper and lower bounds of the algorithm and proved the global convergence. Numerical experiments had also verified our theory, and the algorithm is effective for both online and offline data. We found that choosing different initial vectors will affect the convergence speed, and the initial vector could converge to the second or third eigenvectors by satisfying some exceptional conditions.
KeywordsGALR PCA AlgorithmDDT MethodGlobal ConvergenceOnline Data
- Bouwmans, T., Javes, S., et al. (2018) On the Applications of Robust PCA in Image and Video Processing. Proceedings of the IEEE, 106, 1427-1457. https://doi.org/10.1109/JPROC.2018.2853589
- Oja, E. (1982) Simplified Neuron Model as a Principal Component Analyzer. Journal of Mathematical Biology, 15, 267-273. https://doi.org/10.1007/BF00275687
- Xu, L. (1993) Least Mean Square Error Reconstruction Principle for Self-Organizing Neural-Nets. Neural Networks, 6, 627-648. https://doi.org/10.1016/S0893-6080(05)80107-8
- Chatterjee, C., Kang, Z. and Roychowdhury, V.P. (2000) Algorithms for Accelerated Convergence of Adaptive PCA. IEEE Transactions on Neural Networks, 11, 338-355. https://doi.org/10.1109/72.839005
- Xu, L. and Yuille, A.L. (1995) Robust Principal Component Analysis by Self-Organizing Rules Based on Statistical Physics Approach. IEEE Transactions on Neural Networks, 6, 131-143. https://doi.org/10.1109/72.363442
- Wang, S., Liang, Y.L. and Ma, F. (1998) An Adaptive Robust PCA Neural Network. The 1998 IEEE International Joint Conference on Neural Networks Proceedings, Vol. 3, 2288-2293. https://doi.org/10.1109/IJCNN.1998.687218
- Yang, T.-N. and Wang, S.D. (1999) Robust Algorithms for Principal Component Analysis. Pattern Recognition Letters, 20, 927-933. https://doi.org/10.1016/S0167-8655(99)00060-4
- Chen, T.P., Hua, Y.B. and Yan, W.-Y. (1998) Global Convergence of Oja’s Subspace Algorithm for Principal Component Extraction. IEEE Transactions on Neural Networks, 9, 58-67. https://doi.org/10.1109/72.655030
- Zhang, Q. and Bao, Z. (1995) Dynamical Systems for Computing the Eigenvectors Associated with the Largest Eigenvalue of a Positive Definite Matrix. IEEE Transactions on Neural Networks, 6, 790-791. https://doi.org/10.1109/72.377989
- Zhang, Q. and Leung, Y.-W. (2000) A Class of Learning Algorithms for Principal Component Analysis and Minor Component Analysis. IEEE Transactions on Neural Networks, 11, 529-533. https://doi.org/10.1109/72.839022
- Zhang, Q.F. (2003) On the Discrete-Time Dynamics of a PCA Learning Algorithm. Neurocomputing, 55, 761-769. https://doi.org/10.1016/S0925-2312(03)00439-9
- Zufiria, P.J. (2002) On the Discrete-Time Dynamics of the Basic Hebbian Neural-Network Nods. IEEE Transactions on Neural Networks, 13, 1342-1352. https://doi.org/10.1109/TNN.2002.805752
- Yi, Z., Ye, M., Lv, J.C. and Tan, K.K. (2005) Convergence Analysis of a Deterministic Discrete Time System of Oja’s PCA Learning Algorithm. IEEE Transactions on Neural Networks, 16, 1318-1328. https://doi.org/10.1109/TNN.2005.852236