Unsupervised Multi-Level Non-Negative Matrix Factorization Model: Binary Data Case
- 1 Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, USA
- 2 School of Information Engineering, Wuhan University of Technology, Wuhan, China
- 3 Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, USA
- 4 Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, USA
- 5 Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, USA
Abstract
Rank determination issue is one of the most significant issues in non-negative matrix factorization (NMF) research. However, rank determination problem has not received so much emphasis as sparseness regularization problem. Usually, the rank of base matrix needs to be assumed. In this paper, we propose an unsupervised multi-level non-negative matrix factorization model to extract the hidden data structure and seek the rank of base matrix. From machine learning point of view, the learning result depends on its prior knowledge. In our unsupervised multi-level model, we construct a three-level data structure for non-negative matrix factorization algorithm. Such a construction could apply more prior knowledge to the algorithm and obtain a better approximation of real data structure. The final bases selection is achieved through L 2 -norm optimization. We implement our experiment via binary datasets. The results demonstrate that our approach is able to retrieve the hidden structure of data, thus determine the correct rank of base matrix.
- D. D. Lee and H. S. Seung, “Learning the Parts of Objects by Non-Negative Matrix Factorization,” Nature, Vol. 401, No. 6755, 1999, pp. 788-791.
- Z. Yuan, E. Oja, “Projective Non-negative Matrix Factorization for Image Compression and Feature Extraction,” Springer, Heidelberg, 2005.
- C. Fevotte, N. Bertin and J. L. Durrieu, “Non-Negative Matrix Factorization with the Itakura-Saito Divergence,” With Application to Music Analysis. Neural Computation, Vol. 21, No. 3, 2009, pp. 793-830.
- M. W. Berry and M. Browne, “Email Surveillance Using Non-Negative Matrix Factorization,” Computational and Mathematical Organization Theory, Vol. 11, No. 3, 2005, pp. 249-264.
- Q. Sun, F. Hu and Q. Hao, “Context Awareness Emergence for Distributed Binary Pyroelectric Sensors,” Proceeding of 2010 IEEE Conference on Multisensor Fusion and Integration for Intelligent Systems, Salt Lake City, 5-7 September 2010, pp.162-167.
- F. Hu, Q. Sun and Q. Hao, “Mobile Targets Region-of- Interest via Distributed Pyroelectric Sensor Network: Towards a Robust, Real-Pyroelectric Sensor Network,” Proceeding of 2010 IEEE Conference on Sensors, Waikoloa, 1-4 November 2010, pp. 1832-1836.
- Y. Xue, C. S. Tong, Y. C. W. Chen, “Clustering-Based Initialization for Non-negative Matrix Factorization,” Applied Mathematics and Computation, Vol. 205, No. 2, 2008, pp. 525-536.
- Z. Yang, Z. Zhu and E. Oja, “Automatic Rank Determination in Projective Non-negative Matrix Factorization,” Proceedings of 9th International Conference on LVA/ICA, St. Malo, 27-30 September 2010, pp. 514-521.
- A. T. Cemgil, “Bayesian Inference for Non-negative Matrix Factorization Models,” Computational Intelligence and Neuroscience, Vol. 2009, 2009, Article ID 785152.
- M. Said, D. Brie, A. Mohammad-Djafari and C. Cedric, “Separation of Nonnegative Mixture of Nonnegative Sources using a Bayesian Approach and MCMC Sampling,” IEEE Transactions on Signal Processing, Vol. 54, No. 11, 2006, pp. 4133-4145.