Adjusting the penalized term for the regularized regression models

Abstract

More attention has been given to regularization methods in the last two decades as a result of exiting high-dimensional ill-posed data. This paper proposes a new method of introducing the penalized term in regularized regression. The proposed penalty is based on using the least squares estimator’s variances of the regression parameters. The proposed method is applied to some penalized estimators like ridge, lasso, and elastic net, which are used to overcome both the multicollinearity problem and selecting variables. Good results are obtained using the average mean squared error criterion (AMSE) for simulated data, also real data are shown best results in the form of less average prediction errors (APE) of the resulting estimators.

Keywords: Elastic-Net, Lasso, Penalized regression; Regularization; Ridge regression; Shrinkage; Variable selection

AMS 2010 Mathematics Subject Classification : 62J05; 62J07