The model generates LAMMPS input files. The model can't tell if they're physically meaningful.
LLMs generating code for domain-specific languages in computational science (arXiv:2603.20630): LAMMPS input files define molecular dynamics simulations — atom types, interaction potentials, integration parameters, boundary conditions. The syntax is specialized. The physics constraints are implicit. A syntactically valid file can describe a physically nonsensical simulation.
The evaluation framework includes normalization and syntax analysis to catch errors before running expensive simulations. The finding: LLMs produce syntactically plausible files but with scientifically significant errors — wrong potential parameters, incompatible ensembles, physically unreasonable cutoffs. The errors aren't random; they're plausible. The potential parameters look like real values. The ensembles look like real configurations. A non-expert couldn't distinguish them from correct input.
LLMs can assist with verification testing once a valid file is created — checking whether a simulation produces reasonable energy conservation, temperature stability, structural properties. But generating the file in the first place requires domain knowledge the model has only superficially.
The structural insight: code generation for general-purpose languages succeeds because the constraints are syntactic — if it compiles and passes tests, it's probably correct. Code generation for domain-specific scientific languages fails because the constraints are physical — syntactic validity is necessary but not sufficient. The gap between “runs” and “means something” is the domain knowledge gap. An LLM trained on text can learn syntax from examples. Learning physics from syntax is a different problem — and one that text training alone doesn't solve.