Optimization of Skin Disease Classification with a Hybrid Convolutional Neural Network and Vision Transformer Approach

Abstract

Skin diseases remain a significant global health concern, with accurate diagnosis often challenged by visual similarities among lesion types and limited access to specialist dermatological services. This study presents an optimized hybrid Convolutional Neural Network-Vision Transformer (CNN-ViT) framework for automated skin disease classification using dermoscopic images from the HAM10000 dataset. The proposed architecture combines the local feature extraction capability of CNNs with the global contextual learning ability of Vision Transformers to improve classification performance. Prior to model training, the dataset was partitioned into training (70%), validation (15%), and testing (15%) subsets, with data augmentation applied exclusively to the training set to address class imbalance and improve generalization. The hybrid model was implemented using TensorFlow/Keras and evaluated using weighted accuracy, precision, recall, and F1-score metrics. Experimental results demonstrated a classification accuracy of 97.68%, precision of 89.80%, recall of 90.20%, and F1-score of 91.30%. Comparative analysis indicated that the hybrid architecture outperformed conventional CNN-based approaches due to its ability to simultaneously capture local texture patterns and global lesion structures. The findings demonstrate the effectiveness of integrating CNN and ViT architectures for intelligent dermatological image analysis and highlight the potential of hybrid deep learning models in supporting clinical decision-making and early skin disease diagnosis.