Standard M-estimators — the broad class including maximum likelihood, least squares, and robust estimators — are the workhorses of statistics. They are defined as solutions to optimization problems: find the parameter that minimizes a loss function over the data. The assumption is that the solution exists. The question is how good it is.
The paper shows the solution can simply not exist. In single index models at high dimension, there is a sharp phase transition: below a critical ratio of sample size to dimension, the unregularized M-estimator has no solution. Not a bad solution, not a noisy solution — no solution at all. The loss function has no finite minimizer. The optimization problem is unbounded.
The transition is abrupt. Above the threshold, the estimator exists and is well-behaved. Below it, the estimator ceases to exist entirely. There is no gradual degradation, no warning sign in the curvature of the loss function at moderate sample sizes. The estimator is either there or it isn't.
Regularization (adding a penalty term) forces a solution to exist by bounding the search space. This is the standard practice, but it is typically motivated as a way to improve statistical performance. The paper reveals a more fundamental role: regularization is not just helpful but necessary — below the threshold, it is the only thing that makes the problem well-defined.
The structural point: the existence of a statistical estimator is not guaranteed by the model's specification but depends on having enough data relative to the dimension. The cliff between “estimator exists” and “estimator does not exist” is as sharp as a physical phase transition. The gradual picture — more data means better estimates — is wrong at the boundary. More data means the estimate starts existing.