Research ArticleOpen AccessGoogle Scholar indexed
FPGA Implementation of a Scalable and Highly Parallel Architecture for Restricted Boltzmann Machines
Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Japan
Swiss Federal Institute of Technology, Lausanne, Switzerland
Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Japan
Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Japan
Swiss Federal Institute of Technology, Lausanne, Switzerland
- 1 Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Japan
- 2 Swiss Federal Institute of Technology, Lausanne, Switzerland
- 3 Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Japan
- 4 Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Japan
- 5 Swiss Federal Institute of Technology, Lausanne, Switzerland
Circuits and Systems·Volume 07 (2016)·Pages 2132–2141·Published 5 July 2016·DOI10.4236/cs.2016.79185
Copy link · social · email
Abstract
Restricted Boltzmann Machines (RBMs) are an effective model for machine learning; however, they require a significant amount of processing time. In this study, we propose a highly parallel, highly flexible architecture that combines small and completely parallel RBMs. This proposal addresses problems associated with calculation speed and exponential increases in circuit scale. We show that this architecture can optionally respond to the trade-offs between these two problems. Furthermore, our FPGA implementation performs at a 134 times processing speed up factor with respect to a conventional CPU.
KeywordsDeep LearningRestricted Boltzmann Machines (RBMs)FPGAAcceleration
- Hinton, G. and Salakhutdinov, R.R. (2006) Reducing the Dimensionality of Data with Neural Networks. Science, 313, 504-507. http://dx.doi.org/10.1126/science.1127647
- Hinton, G., Osindero, S. and Teh, Y. (2006) A Fast Learning Algorithm for Deep Belief Nets. Neural Computation, 18, 1527-1554. http://dx.doi.org/10.1162/neco.2006.18.7.1527
- LeCun, Y., Bengio, Y. and Hinton, H. (2015) Deep Learning. Nature, 521, 436–444. http://dx.doi.org/10.1038/nature14539
- Kim, S., McMahon, P. and Olukotun, K. (2010) A Large-Scale Architecture for Restricted Boltzmann Machines. Proceedings of the 18th IEEE Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), Charlotte, NC, 2-4 May 2010, 201-208. http://dx.doi.org/10.1109/fccm.2010.38
- Ly, D. and Chow, P. (2010) High-Performance Reconfigurable Hardware Architecture for Restricted Boltzmann Machines. IEEE Transactions on Neural Networks, 21, 1780-1792. http://dx.doi.org/10.1109/TNN.2010.2073481
- Hinton, H. (2002) Training Products of Experts by Minimizing Contrastive Divergence. Neural Computation, 14, 1771-1800. http://dx.doi.org/10.1162/089976602760128018
- Welling, M. and Sutton, C. (2005) Learning in Markov Random Fields with Contrastive Free Energies. Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics (AISTATS), Barbados, 6-8 January 2005, 397- 404.
- Salakhutdinov, R.R., Mnih, A. and Hinton, G. (2007) Restricted Boltzmann Machines for Collaborative Filtering. Proceedings of the International Conference on Machine Learning, 24, 791-798. http://dx.doi.org/10.1145/1273496.1273596
- Fischer, A. and Igel, C. (2014) Training Restricted Boltzmann Machines: An Introduction. Pattern Recognition, 47, 25-39. http://dx.doi.org/10.1016/j.patcog.2013.05.025
- Nitta, Y. and Nakamura, K. (1989) US Patent 4,890,251. Washington, DC: U.S.
- LeCun, Y., Bottou, L.,Bengio, Y. and Haffner. P. (1998) Gradient-Based Learning Applied to Document Recognition. Proceedings of the IEEE, 86, 2278-2324. http://dx.doi.org/10.1109/5.726791
- Kim, S.K., MacAfee, L.C., McMahon, P.L. and Olukotun, K. (2009) A Highly Scalable Restricted Boltzmann Machine FPGA Implementation. Proceedings the International Conference on Field Programmable Logic and Applications, Prague, 31 August 2009-2 September 2009, 367-372. http://dx.doi.org/10.1109/fpl.2009.5272262