The softmax function — exponentiate each input, divide by the sum — is the bottleneck of transformer inference. It's nonlinear, requires global normalization, and must be computed for every attention head at every layer. In electronic hardware, it's fast enough. In photonic hardware, where the goal is to run inference at the speed of light, the exponential function is the problem. Light doesn't naturally exponentiate.
Li et al. (arXiv:2603.12934) build the exponential from cascaded microring resonators in thin-film lithium niobate (TFLN). Each microring has a Lorentzian transfer function — a resonance peak that maps input signal to output intensity. Cascading multiple microrings multiplies the transfer functions, and the product of Lorentzians can approximate an exponential over a useful input range.
The key design parameter is the number of cascaded stages. More rings give a better exponential approximation but consume more chip area and introduce more insertion loss. The authors characterize the tradeoff: three to five cascaded microrings approximate the exponential well enough for softmax in transformer attention, with error small relative to the quantization noise already present in reduced-precision inference.
The approach sidesteps the usual problem with optical nonlinearities: they're either too weak (Kerr effects need high power) or too strong (saturable absorbers are binary). The microring cascade creates an effective nonlinearity from cascaded linear resonances. Each stage is linear; the composition is nonlinear. The exponential isn't computed — it's constructed from the physics of resonant coupling.
For photonic AI accelerators, this is the missing piece. Matrix multiplication in optics is mature (Mach-Zehnder meshes, wavelength-division multiplexing). The activation function was the holdout. If the exponential can be done in photonics, the entire attention mechanism stays in the optical domain. No electro-optical conversion between the linear and nonlinear steps.