The Lasso selects a subset of features by penalizing the L1 norm of coefficients. In the proportional regime — where the number of features p is comparable to the sample size n — the Lasso selects an “active set” of nonzero coefficients. The hope: the active set is small (sparse), and the corresponding predictions are accurate.
The paper proves an if-and-only-if: the Lasso prediction error is bounded if and only if the active set size stays strictly below n. Not approximately below — strictly below. The moment the active set size reaches n, the prediction error becomes unbounded. The condition is the sharp boundary between a useful estimator and a useless one.
The direction of the equivalence matters. It was known that small active sets imply bounded error (sparse solutions are good solutions). The converse was not known: bounded error implies small active sets. The paper establishes that there is no way to achieve bounded prediction error with a near-saturated active set. If the Lasso is using almost all its degrees of freedom, it is necessarily overfitting.
This connects two seemingly different questions — “how many features does the Lasso select?” and “how well does the Lasso predict?” — into one. They are not two measurements of the same estimator; they are the same measurement in different units. Active set size and prediction error are coupled by an equivalence, not merely a correlation.
The structural point: the Lasso's success condition is not gradual but binary, determined by a single ratio. The boundary at n is exact — not approximate, not asymptotic. The moment the active set saturates the sample size, the estimator fails. The sparsity is not a nice-to-have; it is the condition for the estimator's existence.