Estimator of prediction error based on AMP
for SCAD and MCP
We propose an approximate message passing-based (AMP-based) estimators of the prediction error for penalized linear regression
[Sakata A. (2018)].
In this paper, we propose two types of estimators denoted by
εpre(1) and
εpre(2).
The explanation of these estimators
and matlab codes for calculating the estimators are as follows:
- Estimator εpre(1):
The estimator εpre(1)
is asymptotically unbiased when the
predictor matrix and data vectors are i.i.d. Gaussian.
We have provided the matlab codes for calculating εpre(1)
under SCAD and MCP here.
For a non-Gaussian case, εpre(1)
is not always an accurate estimator.
When the predictors are correlated,
the behavior of εpre(1) is
similar to that of AIC,
thereby underestimating the prediction error
when we use SCAD and MCP as regularizers.
- Estimator εpre(2):
The estimator εpre(2)
is a corrected version of εpre(2),
which considers the correlation between predictors.
We have provided the matlab codes for calculating εpre(2)
under SCAD and MCP here.
We applied our estimator to "Communities and Crime Unnormalized Data Set",
redefining the predictors to decrease the correlation between predictors.
The modified data are contained in the zip file downloaded from the above link.
The detail of the numerical simulation is explained in [Sakata A. (2018)].