Standard reinforcement learning for portfolio optimization maximizes discounted expected returns. The agent values a dollar tomorrow less than a dollar today, with the discount rate fixed. This is computationally tractable but psychologically unrealistic — human investors care about risk in ways that simple discounting can't capture.
The paper on portfolio optimization under recursive utility (arXiv: 2603.22880) replaces the standard objective with recursive utility from asset-pricing theory. In this framework, the agent's valuation of future outcomes depends on a certainty equivalent that has no closed form under observed returns. The authors approximate it by K-sample Monte Carlo and train actor-critic algorithms (PPO, A2C) on the resulting value targets.
The key technical contribution is an approximate advantage estimate (AAE) that generalizes the Bellman residual to multi-step with state-dependent weights. This formulation applies only to critic-based algorithms — policy-gradient-only methods can't use it.
On ten chronological train/test splits of South Korean ETF data, the recursive-utility agent improves on the discounted baseline in Sharpe ratio, maximum drawdown, and cumulative return.
The through-claim: the objective function matters more than the algorithm. The same RL architecture with a risk-sensitive objective outperforms the same architecture with a naive one. The improvement comes not from better learning but from better preferences — the agent optimizes for something closer to what a risk-aware investor actually wants. The recursive utility doesn't add information; it changes what counts as success.
2603.22880. Quantitative finance / reinforcement learning / recursive utility / portfolio optimization / risk sensitivity.