A 3D printer uses filament from reels. Each print job consumes some length. When a reel doesn't have enough filament left for the next job, it's discarded — the remaining filament becomes waste. The printer can only hold a limited number of reels simultaneously. Jobs arrive online, their sizes unknown in advance. Which reel should receive each job?
Celenk, van Jaarsveld, Adan, and Akcay (arXiv:2603.23042) recognize this as a variant of the online bin packing problem, formulated as an infinite-horizon average-cost Markov Decision Process. The objective: minimize the long-run average discarded filament.
Under random allocation, the system decomposes into identical single-reel processes — each reel evolves independently, and the waste can be computed in closed form. This isn't the solution; it's the baseline. But the decomposition reveals the structure. The marginal cost of assigning a job to a specific reel depends on how much filament remains — assigning a small job to an almost-empty reel wastes less than assigning it to a nearly full one, because the nearly full reel can still serve larger jobs.
This produces an index policy: score each reel by the marginal cost of using it, assign the job to the cheapest reel. The index policy provably outperforms random allocation. Integrating it into deep reinforcement learning via approximate policy iteration achieves near-optimal performance while maintaining real-time feasibility.
The structural insight is that the online problem has enough structure to be partially solved analytically (the decomposition, the index), and this analytical structure guides the learning rather than replacing it. The closed-form baseline tells the RL agent what good looks like; the agent learns the refinement. Structure and learning are complementary, not competing.