Reducing Data Chaos and Partitioning the Training Sample into Macro-Features in Classification Problem
- 1 St. Petersburg, Russia
Abstract
This paper is devoted to revealing some features of machine learning problems and developing a new approach to solving them. It is based on the application of information processing technology by animal sensory systems, each of which perceives information of only a certain type. Therefore, computational operations for solving the classification problem are performed mainly for individual features of objects, although they are usually carried out for objects as a whole. This approach ensures the simplicity of the algorithm and the ability to order the features by sorting in non-decreasing order their values, which leads to a decrease in the entropy and chaos of the data. It has been established that ordered features are hidden variables that allow us to detect the functional relationship “feature-class” and to partition any training sample into macro-features, which are ordered features of objects of a certain class. Classification of any object in test sample comes down to calculating the frequency of occurrence of its feature values in the nearest neighborhood of ordered feature values of the corresponding macro-feature of a certain class. The object class corresponds to the maximum of the average value of this frequency. Applying of ordered features opens up the prospect of new types of neural networks. The article also discusses the application of an ordered data matrix to solve problems of partitioning a set into clusters of objects with common properties.
- Luger, G.F. (2016) Artificial Intelligence: Structures and Strategies for Complex Problem Solving. 6th Edition, Addison-Wesley.
- Solso, R. (2006) Cognitive Psychology. 6th Edition, Allyn and Bacon, 589.
- Shats, V.N. (2018) The Classification of Objects Based on a Model of Perception. In: Kryzhanovsky, B., et al ., Eds., Advances in Neural Computation , Machine Learning , and Cognitive Research , Springer International Publishing, 125-131. https://doi.org/10.1007/978-3-319-66604-4_19
- Smith, C.U.M. (2004) Biology of Sensory Systems. John Wiley and Sons Limited, 565.
- Shats, V.N. (2022) Properties of the Ordered Feature Values as a Classifier Basis. Cybernetics and Physics , 11, 25-29. https://doi.org/10.35470/2226-4116-2022-11-1-25-29
- Hastie, T., Tibshirani, R. and Friedman, R. (2009) The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd Edition, Springer, 764.
- Shats, V.N. (2024) Feature Ordering as a Way to Reduce the Entropy of the Training Sample and the Basis of the Simplest Classification Algorithms. Proceeding 26 th International Conference Neuroinformatics , Moskow, 24-26 October 2024, 164-173.
- Grenander, U. (1976) Lectures on Pattern Theory 1: Pattern Synthesis. Springer-Verlag.
- Feynman, R. (1965) The Character of Physical Law. A Series of Lectures Recorded by the BBC at Cornell University USA. Cox and Wyman.
- Prigogine, I. and Stengers, I. (1984) Order Out of Chaos: Men’s New Dialogue with Nature. Flamingo Edition.
- Andrievskii, B.R. and Fradkov, A.L. (2003) Control of Chaos: Methods and Applications. I. Methods. Automation and Remote Control , 64, 673-713. https://doi.org/10.1023/a:1023684619933
- David, H.A. and Nagaraja, H.N. (2003) Order Statistics. 3rd Edition, Wiley. https://doi.org/10.1002/0471722162
- Kolmogorov, A.N. and Fomin, S.V. (1957) Elements of the Theory of Functions and Functional Analysis. Vol. 1 Metric and Normed Spaces. Graylock Press.
- Asuncion, A. and Newman, D. (2007) UCI Machine Learning Repository. Irvine University of California.
- Shats, V.N. (2023) Principle Splitting of Finite Set in Classification Problem. Proceeding 25 th International Conference Neuroinformatics , Moskow, 23-27 October 2023, 262-270.