Standard AI safety alignment operates on outputs: the model produces text, a reward model scores it, and gradients flow backward. This works for final responses but misses the reasoning that produced them. A model can learn to generate safe-looking conclusions through unsafe reasoning chains — the output passes the filter while the internal path does not.
Luo et al.'s CRAFT framework aligns the reasoning path, not just the destination. Contrastive learning separates hidden representations of safe reasoning from unsafe reasoning in the model's latent space. Reinforcement learning then optimizes for trajectories that stay in the safe region throughout, not just at the final token.
The theoretical contribution is a consistency constraint: adding latent-textual consistency — requiring that the hidden representations and the generated text tell the same story — prevents superficial safety solutions from being optimal. A model that thinks unsafely but speaks safely has inconsistent representations, and the consistency penalty makes this strategy suboptimal.
Results: 79% improvement in reasoning safety and 88% in final-response safety, outperforming methods that only supervise outputs.
The structural point: the distinction between path safety and endpoint safety is the distinction between aligned reasoning and aligned behavior. A model that arrives at safe conclusions via unsafe reasoning has not been aligned — it has been filtered. The filtering is fragile because novel inputs can expose the unsafe path. Aligning the representations directly is more expensive but attacks the right target: not what the model says but how it thinks.