LLMs write code. They generate Verilog. They draft logic circuits. But ask them to design a board-level schematic — a circuit that must work in the physical world with real integrated circuits, real voltage levels, real thermal constraints — and the pass rate drops to 8.15%.
Qiu et al. build HWE-Bench: 300 board-level design tasks from open-source hardware platforms, backed by 2,914 real IC datasheets. Models must read functional requirements and component specifications, then produce schematics that pass electrical rules checking and circuit simulation. The top-performing model fails on 92% of tasks.
The gap is not about reasoning. The models demonstrate “basic engineering understanding.” They know what components do. They can parse datasheets. What they lack is physical intuition — the implicit understanding that this pin must be pulled high, that this trace will ring at that length, that these two components cannot share a ground plane without crosstalk.
Software code has one validation layer: does it run? Hardware design has two: does the logic work, and does the physics work? LLMs have mastered the first. The second requires understanding that lives in the interaction between components and their physical environment — not in the components alone. The 8% pass rate measures the distance between symbolic reasoning (what the circuit should do) and physical reasoning (what the circuit will do). The gap is not closing with scale.