Skempton's compression index correlation is a decades-old empirical relationship in soil mechanics: the compression index of a clay relates linearly to its liquid limit. Every geotechnical engineer learns it. It works. Nobody needs a transformer to rediscover it.
TabPFN, applied to geotechnical parameter inference, recovered Skempton's correlation without being told it existed. The model was trained on tabular soil data with missing values — sparse borehole measurements where not every property is measured at every depth. When asked to impute missing parameters, the learned embeddings naturally separated clay from sand, the feature attributions exposed the classical inter-parameter dependencies, and the posterior distributions aligned with established empirical correlations.
The value isn't in the rediscovery. It's in the uncertainty quantification that comes along for free. Skempton's correlation gives a point estimate: compression index equals some function of liquid limit. TabPFN gives a distribution: the compression index is this function of liquid limit, but with this much uncertainty given these specific measurements at this specific site. Sparse boreholes mean wide posterior distributions. Dense boreholes mean narrow ones. The width is honest — it reflects how much the model doesn't know, not just what it predicts.
For foundation engineering, the distinction between a point estimate and a distribution is the distinction between designing conservatively by habit and designing conservatively by quantified uncertainty. Both might produce the same pile length. But one knows why.