The 55th Statistical Machine Learning Seminar (2023.3.17)
This is an online seminar. Please register at the following Google Forms. A Zoom link will be emailed to you. This seminar is sponsored by "the 2022 Osaka Metropolitan University (OMU)Strategic Research Promotion Project (STEP-UP Research)" (Yoshihiko Konno (OMU)) and by Research Center for Statistical Machine Learning in The Institute of Statistical Mathematics.
Time:
March 17 (Fri), 2023. 13:00 - 14:30
Place:
Zoom (online)
Seminar
Speaker:
Piotr Graczyk (Angers University, France)
Title:Pattern Recovery by Penalized Estimators with Polyhedral Penalty
Abstract: There are many Penalized Least Squared Estimators (LASSO, Fused, Clustered and Generalized LASSO, SLOPE...) intensely used in 21st century Statistical Machine Learning for the recovery of information on the unknown regression parameter \(\beta\) in the regression problem \(Y=X\beta +\varepsilon\). For most of them, the penalty term \(\lambda {\rm pen}(b)\) in the optimization problem $$ \hat\beta:= \mathop{\rm Argmin}_{b \in R^p} \frac{1}{2} \|y - Xb\|_2^2 + \lambda {\rm pen}(b) $$ is a real-valued polyhedral norm or gauge. For LASSO, the penalty is the \(l^1\) norm. It is well known that the LASSO estimator recovers well the sign of the parameter vector \(\beta\), e.g. \(\mathrm{sign}(0, 5,-6, -5, 0,6)=(0, 1,-1, -1, 0,1)\). Any Penalized Least Squared Estimator tends to recover a pattern of the parameter vector \(\beta\). For SLOPE, the penalty is a sorted \(l^1\) norm wih a tuning vector \(\Lambda\in R^p\). It was observed that SLOPE recovers the signed hierarchy \(h(\beta)= \mathrm{rank}(|\beta|)\times \mathrm{sign}(\beta)\), e.g. \(h(0, 5,-6, -5, 0,6)=(0, 1,-2, -1, 0,2)\). Thus SLOPE may eliminate, like LASSO, irrelevant predictors and may identify groups of predictors having the same influence on the vector of responses. We identify the pattern of a Penalized Estimator as a common subdifferential \(\partial {\rm pen}(b)\) of the penalty. This novel definition of a pattern opens a way of mathematical investigations on the pattern recovery by a Penalized Estimator. We give a practical necessary and sufficient condition for SLOPE pattern recovery of an unknown parameter \(\beta\).