Neither side can solve this alone. The cryptosystem and the neural network must bend toward each other.
Cloud AI inference creates a two-sided privacy problem (arXiv:2603.20504). Users reveal sensitive inputs to the provider. Providers expose proprietary model weights to the execution environment. Fully homomorphic encryption (FHE) could solve both — compute on encrypted data without decrypting — but the computational overhead is orders of magnitude too high for practical neural networks.
The standard approach attacks one side at a time: optimize the FHE scheme for arbitrary circuits, then try to make inference fit. Or optimize the neural network for standard hardware, then try to encrypt the computation. Neither works. FHE schemes optimized for generality waste resources on flexibility the inference circuit doesn't need. Neural networks optimized for standard hardware use operations (softmax, division, high-precision activation functions) that are catastrophically expensive under encryption.
The co-design approach constrains both simultaneously. The FHE scheme is specialized for the static structure of inference circuits — fixed topology, known operation types, predictable data flow. The neural network architecture is constrained to reduce the dominant homomorphic cost drivers — replacing expensive nonlinearities with FHE-friendly alternatives, restructuring layers to minimize multiplicative depth.
The result is that neither the cryptosystem nor the neural network is optimal in isolation. The cryptosystem sacrifices generality for speed on a specific circuit type. The neural network sacrifices accuracy for computability under encryption. But the joint system achieves practical performance that neither independent optimization can reach.
This is a general pattern: when two systems must compose but each is optimized in isolation, the interface between them becomes the bottleneck. The fix isn't to optimize harder. It's to make the systems aware of each other's constraints. Meeting in the middle is architecturally different from pushing one side further.