AISM 54, 900-917

© 2002 ISM

## Improving penalized least squares through adaptive selection of penalty and shrinkage

### Rudolf Beran

Department of Statistics, University of California, Davis, Davis, CA 95616, U.S.A.

(Received February 7, 2001; revised July 2, 2001)

Abstract.
Estimation of the mean function in nonparametric
regression is usefully separated into estimating the means at the observed
factor levels — a one-way layout problem — and interpolation between the
estimated means at adjacent factor levels. Candidate penalized least
squares (PLS) estimators for the mean vector of a one-way layout are
expressed as shrinkage estimators relative to an orthogonal regression
basis determined by the penalty matrix. The shrinkage representation of
PLS suggests a larger class of candidate monotone shrinkage (MS)
estimators. Adaptive PLS and MS estimators choose the shrinkage vector and
penalty matrix to minimize estimated risk. The actual risks of
shrinkage-adaptive estimators depend strongly upon the economy of the
penalty basis in representing the unknown mean vector. Local annihilators
of polynomials, among them difference operators, generate penalty bases
that are economical in a range of examples. Diagnostic techniques for
adaptive PLS or MS estimators include basis-economy plots and estimates of
loss or risk.

Key words and phrases:
Nonparametric regression, one-way layout, adaptation, loss estimator, risk estimator, economical basis, orthogonal polynomial, local annihilator.

**Source**
(TeX , DVI )