Statistics seminar

Abstract: I will present recent results obtained in [1] and [2] jointly with M. Bogdan, X. Dupuis, B. Kolodziejek, U. Schneider, T. Skalski, P. Tardivel and M. Wilczynski.

It is well known that LASSO discovers zero coefficients of the vector bbb in the regression equation Y=Xb+εY=Xb+\varepsilonY=Xb+ε where XXX is the data matrix and YYY the response vector. In fact LASSO estimates the sign of the coefficient vector bbb (bi b_ibi​'s positive, negative or null). The sign is called the model(pattern) of LASSO. In the LASSO estimator the ℓ1\ell^1ℓ1 penalty is employed.

In the study of Big Data one needs to identify more informative patterns of the vector bbb. These leads to use penalties different from the ℓ1\ell^1ℓ1 penalty and to get more dimensionality reduction.

We define the pattern of any estimator with polyhedral penalty, i.e. the unit ball BBB with respect to the penalty norm is a convex polyhedron. Surprising links between the pattern of a penalized estimator and the geometry of the convex polytope B∗B^*B∗ will be explained.

We study in detail estimation with a sorted ℓ1\ell^1ℓ1 penalty, called SLOPE. Its dual ball B∗B^*B∗ is a signed permutahedron. SLOPE is a popular method for dimensionality reduction in the high-dimensional regression, encompassing the LASSO estimator but also the l∞l^\inftyl∞ penality. Indeed, some coefficients of the estimator b^SLOPE\hat b ^{\rm SLOPE}b^SLOPE are null (sparsity) and others are equal in absolute value (clustering). Consequently, irrelevant predictors are eliminated and groups of predictors having the same influence on the response vector are identified. The SLOPE pattern of a vector bbb provides: the sign of its components, clusters (components equal in absolute value) and clusters ranking.

In our research we give an analytical necessary and sufficient condition for SLOPE pattern recovery of an unknown vector bbb of regression coefficients. Such condition is called Irrepresentability(IR) condition. For any polyhedral penalty we find a geometric IR condition.

[1] P. Graczyk, U. Schneider, T. Skalski, P. Tardivel, A Unified Framework for Pattern Recovery in Penalized and Thresholded Estimation and its Geometry, Journal of Optimization Theory and Applications(2026) vol. 208(1), 1-41.

[2] M. Bogdan, X. Dupuis, P. Graczyk, B. Kolodziejek, T. Skalski, P. Tardivel, M. Wilczynski, Pattern recovery by SLOPE, Applied and Computational Harmonic Analysis 80(2026), 1-25.