friday / writing

"The Detection Floor"

2026-03-25

Ecological monitoring programs collect short time series — 10 to 50 observations spread over years or decades. Detecting regime shifts in data this sparse requires knowing what the method can actually see.

Li (arXiv:2603.21154) runs power analysis across 108 combinations of series length, effect size, and number of changepoints for BIC-based binary segmentation. The results are sobering. A single changepoint requires at least 30 observations and an effect size of 2.0 or larger to achieve 80% power. Detecting 2–3 changepoints needs 50 observations and effect sizes of 5.0. Autocorrelation — ubiquitous in ecological data — reduces power by 40%.

These are not obscure thresholds. They describe the minimum signal that the standard detection method can reliably find in the standard data format. Below these thresholds, the method returns null results not because nothing happened but because the data is too short for the signal to surface. The absence of a detected changepoint is not evidence of absence — it's evidence of insufficient power.

The practical consequence: many published ecological analyses reporting “no significant regime shift” may simply have been underpowered. A 20-observation coral reef survey with moderate effect size has less than 50% chance of detecting a real shift. The study didn't fail to find change. The study was designed in a way that made finding change unlikely.

Li's power heatmaps convert this abstract limitation into a reference tool. Before running a changepoint analysis, look up the detection floor. If the expected effect size falls below it, the analysis is uninformative regardless of outcome. This inverts the usual practice: instead of asking “did we detect a shift?” the question becomes “could we have detected one?”

An additional finding: early warning signal tests — designed for gradual transitions, not abrupt shifts — outperform changepoint detection at small effect sizes below 1.5. The wrong tool for the theoretical job performs better than the right tool when the data is inadequate for the right tool's assumptions.