A PROMETHEE based evaluation of software defect predictors

Abstract

A software defect is an error, flaw, mistake, or fault in a computer program or system that produces incorrect or unexpected results and the process of locating defective modules in software is software defect prediction. Defect prediction in software improves quality and testing efficiency by constructing predictive stand-alone classifier models or by the use of ensembles methods to identify fault-prone modules. Selection of the appropriate set of single classifier models or ensemble methods for the software defect prediction over the years has shown inconsistent results. In previous analysis, inconsistencies exist and the performance of learning algorithms varies using different performance measures. Therefore, there is need for more research in this field to evaluate the performance of single classifiers and ensemble algorithms in software defect prediction. This study assesses the quality of the ensemble methods alongside single classifier models in the software defect prediction using Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE), a multi criteria decision making (MCDM) approach. Using PROMETHEE, the performance of some popular ensemble methods based on 11 performance metrics over 10 public-domain software defect datasets from the NASA Metric Data Program (MDP) repository was evaluated. Noise is removed from the dataset by performing attribute selection. The classifiers and ensemble methods are applied on each dataset; Adaboostgave the best results. Boosted PART comes first followed by Naïve Bayes and then Bagged PART as the best models for mining of datasets.

Keywords: Ensemble; Classification; Software Defect Prediction; PROMETHEE; MCDM