On testing the hypothesis of population stability for credit risk scorecards

Abstract

Scorecards are models used in credit risk modelling. These models segments a population into various so-called \risk buckets" based on the risk  characteristics of the individual clients. Once a scorecard has been developed, the credit provider typically prefers to keep this model in use for an extended period. As a result, it is important to test whether or not the model still ts the population. To this end, the hypothesis of population stability is tested; this hypothesis species that the current proportions of the population in the various risk buckets are the same as was the case at the point in time at which the scorecard was developed. In practice, this assumption is usually tested using a measure known as the population stability index (which corresponds to the asymmetric Kullback-Leibler discrepancy between discrete distributions) together with a well-known rule of thumb. This paper considers the statistical motivation for the use of the population stability index. Numerical examples are provided in order to demonstrate the eect of the rule of thumb as well as other critical values. Although previous numerical studies relating to this statistic are available, the sample sizes are not realistic for the South African credit market. The paper demonstrates that the population stability index has little statistical
merit as either a goodness-of-t statistic to test the hypothesis of population stability or as an intuitive discrepancy measure. As a result, a novel methodology for testing the mentioned hypothesis is proposed. This methodology includes a restatement of the hypothesis to specify a range of \acceptable" deviations from the specied model. An alternative test statistic is also employed as discrepancy measure; this measure has the advantage of having a simple heuristic interpretation in the context of credit risk modelling.

Key words: Goodness-of-t testing, hypothesis testing, population stability, risk analysis.