Assessment of some selected Machine Learning Performance Metrics in the Prediction of type 2 Diabetes

Abstract

The performance of machine learning models is crucial in the healthcare domain, as high-performing models ensure accurate diagnostics, effective treatments, and improved patient outcomes thereby enhancing overall healthcare quality. However, researchers often face uncertainty in selecting the appropriate metrics to evaluate predictive models. Therefore, this research aimed to assess selected performance evaluation metrics used in machine learning applications across three different datasets. This study utilized datasets from three sources and three machine-learning algorithms. Logistic regression (LR), naïve Bayes (NB), and CATBoost (CATB) were the classification algorithms used in this work. With accuracy, the area under the curve (AUC), recall, precision, F1-score, kappa, and the Matthews correlation coefficient (MCC) as metrics, the system was constructed using the Python programming language. The accuracy, AUC, Recall, Precision, F1-Score, Kappa, and MCC of LR, NB, and CATB were 78.27%, 0.7529, 0.1484, 0.5433, 0.2210, 0.1426 and 0.1871; 83.42%, 0.8998, 0.8989, 0.8455, 0.8659, 0.6482 and 0.6656; and 97.57%, 0.9741, 0.9789, 0.9798, 0.9789, 0.9503 and 0.9516, respectively on dataset 3. The study evaluated the effectiveness of commonly used machine learning metrics in predicting type 2 diabetes, highlighting the risks of relying solely on accuracy for model evaluation. The study's findings can help machine learning engineers choose the right assessment metric for a given task.