Prof. Pierre Bellec
Title: How to Generalize Bias and Variance to Convex Regularized Estimators?
Abstract: Convex estimators such as the Lasso, the matrix Lasso and the group Lasso have been studied extensively in the last two decades, demonstrating great success in both theory and practice. However, there are still simple open questions about these estimators, even in the simple linear regression model. We are particularly interested in the following open questions.
1) The bias and variance of linear estimators is easy to define and provide precise insights on the performance of linear estimators. How can bias and variance be generalized to nonlinear convex estimators?
2) The performance guarantees of these estimators require the tuning parameter to be larger than some universal threshold, but the literature is mostly silent about what happens if the tuning parameter is smaller than this universal threshold. How bad is the performance when the tuning parameter is below the universal threshold?
3) The correlations in the design can significantly deteriorate the empirical performance of these nonlinear estimators. Is it possible to quantify the this deterioration explicitly? Is there a price to pay for correlations; in particular, is the performance for correlated designs always worse than that for orthogonal designs?
4) Most theoretical results on the Lasso and its variants rely on conditions on the design matrix. These conditions greatly simplify the proofs and our understanding of these estimators, but it is still unclear whether these conditions are truly necessary of whether they are an artifact of the proofs. Are these conditions actually necessary?
We will provide some general properties of norm-penalized estimators and propose a generalization of the bias and the variance for these nonlinear estimators. These generalizations of bias/variance will hopefully let us answer the above questions.
Biography: Pierre is an assistant professor in the department of statistics and biostatistics at Rutgers since Fall 2016. Prior to that, he obtained his PhD from ENSAE in Paris. His research interests include aggregation of estimators, shape restricted regression, confidence sets, high-dimensional statistics and concentration inequalities.