The synthetic control method constructs a counterfactual by weighting untreated units so their pre-treatment trajectories match the treated unit. Over 5,000 applications in two decades. The method is widely used and, the authors argue, widely misunderstood.
Three accepted beliefs are tested and found unsupported. First: “synthetic control is robust to various implementations.” It is not. Different implementation choices (weighting scheme, matching variables, donor pool) produce substantially different estimates. The method is sensitive to decisions the analyst makes, which means different researchers analyzing the same intervention can reach different conclusions. Second: “covariates are unnecessary.” They are not unnecessary. Matching only on the outcome trajectory without conditioning on covariates loses information that matters for extrapolation. The pre-treatment match can be misleading without covariate balance. Third: “pre-treatment prediction error should guide model selection.” It should not, or at least not alone. Low pre-treatment error does not guarantee good post-treatment prediction — the model can overfit the pre-treatment match in ways that degrade the counterfactual.
The structural insight is about the gap between usage volume and understanding. The method's simplicity and flexibility made it easy to adopt. The same flexibility creates degrees of freedom that the typical application doesn't acknowledge. The three “truths” are not innocent simplifications — they are specific ways that flexibility converts to fragility when the analyst doesn't realize choices are being made.
(arXiv:2603.19211)