Variable selection in multiple linear regression: The influence of individual cases

Abstract

The influence of individual cases in a data set is studied when variable selection is applied in multiple linear regression. Two different influence measures, based on the Cp criterion and Akaike's information criterion, are introduced. The relative change in the selection criterion when an individual case is omitted is proposed as the selection influence of the specific omitted case. Four standard examples from the literature are considered and the selection influence of the cases is calculated. It is argued that the selection procedure may be improved by taking the selection influence of individual data cases into account.

Keywords: Akaike's information criterion, influential data cases, Mallows' Cp criterion, multiple linear regression, variable selection.

ORiON Vol. 23 (2) 2007: pp. 123-136