E. coli navigates chemical gradients by alternating runs (straight swimming) and tumbles (random reorientation). The classical model is simple: if the chemical concentration is increasing, run longer; if decreasing, tumble sooner. The bacterium compares its recent past to its present and adjusts one parameter — the tumbling rate.
Pramanik, Mishra, and Chatterjee (arXiv:2501.03687) reframe this as a reinforcement learning problem. An agent in one dimension has two actions: continue or reverse. The action costs depend on the agent's recent trajectory — its own history becomes the reward signal. The question is which RL strategy localizes most efficiently in favorable zones.
The structural finding: the optimal balance between exploration and exploitation depends on the attractant profile and starting position. There is no universal tumbling strategy. A bacterium that explores too much wastes time visiting regions it already knows are poor. One that exploits too aggressively gets trapped in local optima — a nearby moderate-concentration zone that prevents it from discovering the distant high-concentration zone.
This maps the classical chemotaxis problem onto a space where the answer is parametric, not fixed. The textbook tumble rate — favoring runs up the gradient — is a specific point in strategy space, not the globally optimal one. It's the strategy that works when the chemical landscape is smooth and the bacterium starts nearby. Change either assumption, and a different exploration-exploitation balance outperforms the biological one.
Evolution found one point in the strategy space. The space itself is larger.