Training with Input Selection and Testing (TWIST) Algorithm: A Significant Advance in Pattern Recognition Performance of Machine Learning — Oak Academic Publishing
Research ArticleOpen AccessGoogle Scholar indexed
Training with Input Selection and Testing (TWIST) Algorithm: A Significant Advance in Pattern Recognition Performance of Machine Learning
Semeion Research Center of Sciences of Communication, Rome, Italy
,
Semeion Research Center of Sciences of Communication, Rome, Italy
,
Department of Mathematical and Statistical Sciences, University of Colorado, Denver, USA.
1 Semeion Research Center of Sciences of Communication, Rome, Italy
2 Semeion Research Center of Sciences of Communication, Rome, Italy
3 Department of Mathematical and Statistical Sciences, University of Colorado, Denver, USA.
This article shows the efficacy of TWIST, a methodology for the design of training and testing data subsets extracted from given dataset associated with a problem to be solved via ANNs. The methodology we present is embedded in al gorithms and actualized in computer software. Our methodology as implemented in software is compared to the current standard methods of random cross validation: 10-Fold CV, random split into two subsets and the more advanced T&T. For each strategy, 13 learning machines, representing different families of the main algorithms, have been trained and tested. All algorithms were implemented using the well-known WEKA software package. On one hand a falsification test with randomly distributed dependent variable has been used to show how T&T and TWIST behaves as the other two strategies: when there is no information available on the datasets they are equivalent. On the other hand, using the real Statlog (Heart) dataset, a strong difference in accuracy is experimentally proved. Our results show that TWIST is superior to current methods. Pairs of subsets with similar probability density functions are generated, without coding noise, according to an optimal strategy that extracts the most useful information for pattern classification.
T. G. Dietterich, “Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms,” Neural Computation, Vol. 10, No. 7, 1998, pp. 1885-1924. doi:10.1162/089976698300017197
M. Buscema, “Genetic Doping Algorithm (GenD): Theory and Application,” Expert Systems, Vol. 21, No. 2, 2004, pp. 63-79.
J. S. Bridle, “Probabilistic Interpretation of Feedforward Classification Network Outputs, with Relationships to Statistical Pattern Recognition,” In: F. Fogelman-Soulié and J. Hérault, Eds., Neuro-Computing: Algorithms, Architectures, Springer-Verlag, New York, 1989.
Y. Chauvin and D. E. Rumelhart, “Backpropagation: Theory, Architectures, and Applications,” Lawrence Erlbaum Associates, Inc. Publishers, Hillsdale, 1995.
D. E. Rumelhart, G. E. Hinton and R. J. Williams, “Learning Internal Representations by Error Propagation,” In: D. E. Rumelhart and J. L. McClelland, Eds., Parallel Distributed Processing, Vol. 1 Foundations, Explorations in the Microstructure of Cognition, The MIT Press, Cambridge, 1986.
M. Buscema., E. Grossi, M. Intraligi, N. Garbagna, A. Andriulli and M. Breda, “An Optimized Experimental Protocol Based on Neuro-Evolutionary Algorithms. Application to the Classification of Dyspeptic Patients and to the Prediction of the Effectiveness of Their Treatment,” Artificial Intelligence in Medicine, Vol. 34, No. 3, 2005, pp. 279-305. doi:10.1016/j.artmed.2004.12.001
S. Penco, E. Grossi, et al., “Assessment of the Role of Genetic Polymorphism in Venous Thrombosis Through Artificial Neural Networks,” Annals of Human Genetics, Vol. 69, No. 6, 2005, pp. 693-706.
E. Grossi, A. Mancini and M Buscema, “International Experience on the Use of Artificial Neural Networks in Gastroenterology,” Digestive and Liver Disease, Vol. 39, No. 3, 2007, pp. 278-285
E. Grossi and M. Buscema, “Introduction to Artificial Neural Networks,” European Journal of Gastroenterology &Hepatology, Vol. 19, No. 12, 2007, pp. 1046-1054. doi:10.1097/MEG.0b013e3282f198a0
E. Grossi, R. Marmo, M. Intraligi and M. Buscema, “Artificial Neural Networks for Early Prediction of Mortality in Patients with Non-Variceal Upper GI Bleeding,” Medical Informatics Insights, Vol. 1, 2008, pp. 7-19.
E. Lahner, M. Intraligi, M. Buscema, M. Centanni, L. Vannella, E. Grossi and B. Annibale, “Artificial Neural Networks in the Recognition of the Presence of Thyroid Disease in Patients with Atrophic Body Gastritis,” World Journal of Gastroenterology, Vol. 14, No. 4, 2008, pp. 563-5688. doi:10.3748/wjg.14.563
S. Penco, M. Buscema, M. C. Patrosso, A. Marocchi and E. Grossi, “New Applicationn of Intelligent Agents in Sporadic Amyotrophic Lateral Sclerosis Identifies Unexpected Specific Genetic Background,” BMC Bioinformatics, Vol. 9, No. 254, 2008. doi:10.1186/1471-2105-9-254
M. E. Street, E. Grossi, C. Volta, E. Faleschini and S. Bernasconi, “Placental Determinants of Fetal Growth: Identification of Key Factors in the Insulin-Like Growth Factor and Cytokine Systems Using Artificial Neural Networks,” BMC Pediatrics, 2008, pp. 8-24..
L. Buri, C. Hassan, G. Bersani, M. Anti, M. A. Bianco, L. Cipolletta, E. Di Giulio, G. Di Matteo, L. Familiari, L. Ficano, P. Loriga, S. Morini, V. Pietropaolo, A. Zambelli, E. Grossi, M. Intraligi, M. Buscema and SIED Appropriateness Working Group, “Appropriateness Guidelines and Predictive Rules to Select Patients for Upper Endoscopy: A Nationwide Multicenter Study,” American Journal of Gastroenterology, Vol. 105, No. 6, 2010, pp. 1327-1337. doi:10.1038/ajg.2009.675
M. Buscema, E. Grossi, M. Capriotti, C. Babiloni and P. M. Rossini, “The I.F.A.S.T. Model Allows the Prediction of Conversion to Alzheimer Disease in Patients with Mild Cognitive Impairment with High Degree of Accuracy, Current Alzheimer Research,” Current Alzheimer Research, Vol. 7, No. 2, 2010, pp. 173-187. doi:10.2174/156720510790691137
F. Pace, G. Riegler, A. de Leone, M. Pace, R. Cestari, P. Dominici, E. Grossi and EMERGE Study Group, “Is It Possible to Clinically Differentiate Erosive from Nonerosive Reflux Disease Patients? A Study Using an Artificial Neural Networks-Assisted Algorithm,” European Journal of Gastroenterology & Hepatology, Vol. 22, No. 10, 2010, pp. 1163-1168.
F. Coppedè, E. Grossi, F. Migheli and L. Migliore, “Polymorphisms in Folate-Metabolizing Genes, Chromosome Damage, and Risk of Down Syndrome in Italian Women: Identification of Key Factors Using Artificial Neural Networks,” BMC Medical Genomics, Vol. 3, No. 42, 2010. doi:10.1186/1755-8794-3-42
G. Rotondano, L. Cipolletta and E. Grossi, “Artificial Neural Networks Accurately Predict Mortality in Patients with Nonvariceal Upper GI Bleeding,” Gastrointestinal Endoscopy, Vol. 73, No. 2, 2011, pp. 218-226. doi:10.1016/j.gie.2010.10.006
F. Brill, D. Brown and W. Martin, “Fast Genetic Selection of Features for Neural Network Classifiers,” IEEE Transactions on Neural Networks, Vol. 3, No. 2, 1992, pp. 324-328. doi:10.1109/72.125874
G. Fung, J. Liu and R. Lau, “Feature Selection in Automatic Signature Verification Based on Genetic Algorithms,” Proceedings of International Conference on Neural Information, Hong Kong Convention and Exhibition Center, 24-27 September 1996, pp. 811-815.
H. Liu and H. Motoda, “Feature Extraction, Construction and Selection: A Data Mining Perspective,” Kluwer Academic Publishers, Boston, 1998. doi:10.1007/978-1-4615-5725-8
N. Chaikla and Y. Qi, “Genetic Algorithms in Feature Selection,” Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Vol. 5, Tokyo, 1999, pp. 538-540.
H. Yuan, S. S. Tseng, W. Gangshan and Z. Fuyan, “A Two-Phase Feature Selection Method Using both Filter and Wrapper,” Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Vol. 2, Tokyo, 1999, pp. 132-136.
M. Kudo and J. Sklansky, “Comparison of Algorithms That Select Features for Pattern Classifiers,” Pattern Recognition, Vol. 33, No. 1, 2000, pp. 25-41. doi:10.1016/S0031-3203(99)00041-2
A. Moser and M. Murty, “On the Scalability of Genetic Algorithms to Very Large-Scale Feature Selection,” Proceedings of Real-World Applications of Evolutionary Computing (EvoWorkshops 2000), Lecture Notes in Computer Science 1803, Springer-Verlag, 2000, pp. 77-86. doi:10.1007/3-540-45561-2_8
A. González and R. Pérez, “Selection of Relevant Features in a Fuzzy Genetic Learning Algorithm,” IEEE Transactions on Systems, Man and Cybernetics. Part B: Cybernetics, Vol. 31, No. 3, 2001, pp. 417-425. doi:10.1109/3477.931534
I. Rivals and L. Personnaz, “Neural Networks Construction and Selection in Nonlinear Modelling,” IEEE Transactions on Neural Networks, Vol. 14, No. 4, 2003, pp. 804-819. doi:10.1109/TNN.2003.811356
Y. Chen and A. Abraham, “Feature Selection and Intrusion Detection Using Hybrid Flexible Neural Tree,” International Symposium on Neural Networks (ISNN2005), Chongqing, 30 May-1 June 2005, pp. 439-444.
P. Leahy, G. Kiely and G. Corcoran, “Structural Optimisation and Input Selection of an Artificial Neural Network for River Level Prediction,” Journal of Hydrology, Vol. 355, 2008, pp. 192-201.
G. Kim and S. Kim, “Feature Selection Using Genetic Algorithms for Handwritten Character Recognition,” Proceedings of the Seventh International Workshop on Frontiers in Handwriting Recognition, Amsterdam, 11-13 September 2000, pp. 103-112.
W. Siedlecki and J. Slansky, “A Note on Genetic Algorithms for Large Scale on Feature Selection,” Pattern Recognition Letters, Vol. 10, No. 5, 1989, pp. 335-347. doi:10.1016/0167-8655(89)90037-8
G. John, R. Kohavi and K. Pfleger, “Irrelevant Features and the Subset Selection Problems,” In: W. Cohen and H. Hirsh, Eds., Machine Learning: Proceedings of the Eleventh International Conference, Morgan Kaufmann Publishers, San Fancisco, 1994, pp. 121-129.
A. Frank and A. Asuncion, “UCI Machine Learning Repository,” University of California, Irvine, 2010. http://archive.ics.uci.edu/ml
M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann and I. H. Witten, “The WEKA Data Mining Software: An Update,” SIGKDD Explorations, Vol. 11, No. 1, 2009, pp. 10-18. doi:10.1145/1656274.1656278
I. H. Witten, E. Frank and M. A. Hall, “Data Mining: Practical Machine Learning Tools and Techniques,” 3rd Edition, Morgan Kaufmann, San Francisco, 2011.
R. O. Duda, P. E. Hart and D. G. Stork, “Pattern Classification,” 2nd Edition, John Wiley and Sons, Inc., New York, 2001.
L. I. Kuncheva, “Combining Pattern Classifiers: Methods and Algorithms,” John Wiley and Sons, Inc., New York.
L. Rokach, “Taxonomy for Characterizing Ensemble Methods in Classification Tasks: A Review and Annotated Bibliography,” Computational Statistics & Data Analysis, Vol. 53, No. 12, 2009, pp. 4046-4072. doi:10.1016/j.csda.2009.07.017
A. Frank and A. Asuncion, “Statlog (Heart) Data Set, UCI Machine Learning Repository,” University of California, School of Information and Computer Science, Irvine, 2010. http://archive.ics.uci.edu/ml/datasets/Statlog+(Heart)