NSU-XGBoost: A Leakage-Controlled XGBoost Framework with Threshold Optimization for Improved Disease Detection

Abstract

Accurate early detection of diabetes is an important objective in clinical decision support and population health analytics. Machine-learning models can improve risk classification; however, apparent gains in performance may be misleading when models are evaluated using different test populations or when information from the test set influences model development. This study presents NSU-XGBoost , a leakage-controlled, SMOTE-enhanced, and hyperparameter-optimized XGBoost framework for diabetes detection. The original dataset of 768 observations was divided into a training cohort of 614 observations and a frozen independent test cohort of 154 observations. The test set remained unchanged and was excluded from all model development, augmentation, and optimization procedures. Synthetic Minority Oversampling Technique (SMOTE) was confined to the training workflow to improve minority-class learning without introducing synthetic observations into the independent evaluation cohort. XGBoost hyperparameters were optimized using five-fold stratified cross-validation within the training data. On the frozen test cohort, tuned XGBoost without augmentation achieved 72.1% accuracy, 51.9% sensitivity, 83.0% specificity, an F1-score of 56.6%, ROC-AUC of 0.818, and PR-AUC of 0.661. NSU-XGBoost improved accuracy to 74.7%, sensitivity to 81.5%, F1-score to 69.3%, ROC-AUC to 0.835, and PR-AUC to 0.725. False-negative classifications decreased from 26 to 10, although specificity declined from 83.0% to 71.0%, demonstrating the expected tradeoff between improved diabetes detection and increased false-positive classifications. The close agreement between cross-validated ROC-AUC and independent-test ROC-AUC indicated no evidence of severe augmentation-induced overfitting. These findings demonstrate that NSU-XGBoost can substantially improve the detection of diabetes-positive cases under a strictly leakage-controlled evaluation framework. The proposed approach emphasizes independent validation, training-only augmentation, and clinically meaningful error analysis rather than reliance on overall accuracy alone.