Analysis of Soft Decision Trees for Passive-Expert Reinforcement Learning
- 1 Department of Mechanical Engineering, The University of Alabama, Tuscaloosa, USA
- 2 Department of Mechanical Engineering, The University of Alabama, Tuscaloosa, USA
Abstract
This paper explores the use of soft decision trees [1] in basic reinforcement applications to examine the efficacy of using passive-expert like networks for optimal Q-Value learning on Artificial Neural Networks (ANN). The soft decision tree networks were built using the PyTorch machine learning and the OpenAi’s Gym environment frameworks. The conducted research study aimed at assessing the performance of soft decision tree networks on Cartpole as provided in the OpenAi Gym software package. The baseline performance metric that the soft decision tree networks were compared against was a simple Deep Neural Network using several linear layers with ReLU and Softmax activation functions for the input and output layers, respectively. All networks were trained using the Backpropagation algorithm provided generically by PyTorch’sAutograd module.
- Frosst, N. and Hinton, G.E. (2017) Distilling a Neural Network into a Soft Decision Tree. ArXiv abs /1711.09784.
- Olden, J.D. and Jackson, D.A. (2002) Illuminating the “Black Box”: A Randomization Approach for Understanding Variable Contributions in Artificial Neural Networks. Ecological Modelling, 154, 135-150. https://doi.org/10.1016/S0304-3800(02)00064-9
- Fahimeh, G., Mehridehnavi, A., Pérez-Garrido, A. and Pérez-Sánchez, H. (2018) Neural Network and Deep-Learning Algorithms Used in QSAR Studies: Merits and Drawbacks. Drug Discovery Today, 23, 1784-1790. https://doi.org/10.1016/j.drudis.2018.06.016
- Shahid, N., Rappon, T. and Berta, W. (2019) Applications of Artificial Neural Networks in Health Care Organizational Decision-Making: A Scoping Review. PLOS ONE, 14, 1-22. https://doi.org/10.1371/journal.pone.0212356
- Castelvecchi, D. (2016) Can We Open the Black Box of AI? Nature, 538, 20-23. https://doi.org/10.1038/538020a
- Dayhoff, J.E. and DeLeo, J.M. (2001) Artificial Neural Networks: Opening the Black Box. Cancer, 91, 1615-1635. https://doi.org/10.1002/1097-0142(20010415)91:8+ 3.0.CO;2-L
- Zdolsek, G., Chen, Y., Bögl, H., Wang, C., Woisetschläger, M. and Schilcher, J. (2001) Deep Networks with Promising Diagnostic Accuracy for the Classification of Atypical Femoral Fractures. Acta Orthopaedica, 92, 1-7. https://doi.org/10.1080/17453674.2021.1891512
- Fernandes, K. and Cardoso, J.S. (2018) Ordinal Image Segmentation Using Deep Neural Networks. 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, 8-13 July 2018, 1-7. https://doi.org/10.1109/IJCNN.2018.8489527
- Mnih, V. (2013) Playing Atari with Deep Reinforcement Learning. ArXiv abs/1312.5602.
- İrsoy, O., Yıldız, T. and Alpaydın, E. (2012) Soft Decision Trees. Proceedings of the 21st International Conference on Pattern Recognition, 1819-1822.
- Hua, Y., Ge, S., Li, C., Luo, Z. and Jin, X. (2018) Distilling Deep Neural Networks for Robust Classification with Soft Decision Trees. 2018 14th IEEE International Conference on Signal Processing (ICSP), Beijing, 12-16 August 2018, 1128-1132. https://doi.org/10.1109/ICSP.2018.8652478
- Hayashi, H. (2013) Neural Network Rule Extraction by a New Ensemble Concept and Its Theoretical and Historical Background: A Review. International Journal of Computational Intelligence and Applications, 12, 1340006. https://doi.org/10.1142/S1469026813400063