Gini Coefficient and AUC in Assessing Predictive Model Performance: Effect of Ranks
- 1 ACA Research Group, Faculty of Business Studies, University of Vaasa, Vaasa, Finland
Abstract
This paper deals with three traditional measures of concentration: (Corrado) Gini coefficient, adjusted Gini coefficient, and AUC. These metrics are popular methods in assessing the performance of predictive models, like credit scoring models. They are non-parametric variables and therefore only depending on the ranking of events. The three measures are closely related to each other. The adjusted Gini coefficient (Accuracy ratio, AR) is only a transformation of the Corrado Gini coefficient being in a linear relationship with AUC. It also equals to Somers’ D . This paper also introduces the measure E , which is based on a classification of the ranks of events. E produces the same result as AUC, but is simple to calculate and interpret. The features of the metrics are discussed in three numerical examples, one of which deals with credit scoring in a large imbalanced dataset (23.533 active firms and 147 bankrupt firms). Numerical examples are used to illustrate the properties of the metrics, especially at the level of ranks.
- Adeodato, P., & Melo, S. (2022). A Geometric Proof of the Equivalence between AUC_ROC and Gini Index Area Metrics for Binary Classifier Performance Assessment. In 2022 International Joint Conference on Neural Networks (IJCNN) (pp. 1-6). IEEE. https://doi.org/10.1109/ijcnn55064.2022.9892048
- Bowles, S., & Carlin, W. (2020). Inequality as Experienced Difference: A Reformulation of the Gini Coefficient. Economics Letters, 186, Article 108789. https://doi.org/10.1016/j.econlet.2019.108789
- Boyd, K., Eng, K. H., & Page, C. D. (2013). Area under the Precision-Recall Curve: Point Estimates and Confidence Intervals. In: H. Blockeel, K. Kersting, S. Nijssen, & F. Železný (Eds.), Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2013 . Springer.
- Cicchitelli, G. (1976). The Sampling Distribution of the Ratio of Two Linear Combinations of Order Statistics. Metron , 34 , 219-228.
- Crook, J. N., Edelman, D. B., & Thomas, L. C. (2007). Recent Developments in Consumer Credit Risk Assessment. European Journal of Operational Research, 183, 1447-1465. https://doi.org/10.1016/j.ejor.2006.09.100
- Gini, C. (1912). Variabilita e mutabilita: Contributo allo studio delle distribuzioni e relazioni statistiche (Vol. III (Part II)). Cuppini.
- Giorgi, G. M., & Gigliarano, C. (2017). The Gini Concentration Index: A Review of The Inference Literature. Journal of Economic Surveys, 31, 1130-1148. https://doi.org/10.1111/joes.12185
- Joshi, R., Gupte, R., & Saravanan, P. (2018). A Random Forest Approach for Predicting Online Buying Behavior of Indian Customers. Theoretical Economics Letters, 8, 448-475. https://doi.org/10.4236/tel.2018.83032
- Kohl, M. (2012). Performance Measures in Binary Classification. International Journal of Statistics in Medical Research, 1, 79-81. https://doi.org/10.6000/1929-6029.2012.01.01.08
- Kumar, P. R., & Ravi, V. (2007). Bankruptcy Prediction in Banks and Firms via Statistical and Intelligent Techniques—A Review. European Journal of Operational Research, 180, 1-28. https://doi.org/10.1016/j.ejor.2006.08.043
- Langohr, H., & Langohr, P. (2009). The Rating Agencies and Their Credit Ratings: What They Are, How They Work, and Why They Are Relevant. John Wiley & Sons, Incorporated.
- Lessmann, S., Baesens, B., Seow, H., & Thomas, L. C. (2015). Benchmarking State-of-the-Art Classification Algorithms for Credit Scoring: An Update of Research. European Journal of Operational Research, 247, 124-136. https://doi.org/10.1016/j.ejor.2015.05.030