Meta Study: RL Post-Training Reduces Agent Retry Success Rates
A new Meta paper reveals that Reinforcement Learning (RL) post-training improves agent performance on the first attempt but significantly degrades their ability to find solutions when given multiple retries. Experiments showed that base models outperformed RL-tuned counterparts in tasks allowing up to 128 attempts, a phenomenon termed the 'Sharpening Tax.' This challenges the assumption that RL universally enhances agentic capabilities, suggesting developers should prioritize long-term coverage over first-try accuracy for retry-capable agents.
Full Report
Rohan Paul (@rohanpaul_ai) · 2026-10-29 17
New Meta paper finds Reinforcement learning post-training makes an agent better on its first attempt and worse at ever finding a solution when it gets 128 attempts.
People assume reinforcement learning teaches agents new skills like tool calling.
That claim comes mostly from math and coding, where base models have already seen huge amounts of data.
They ran 14 base and post-trained model pairs on 3 agentic benchmarks with up to 128 tries per task. They also built PTGS, a sampler that raises temperature on hard prompts and lowers it on easy ones.
On WebShop a gemma-4-31B base model solved over 85% of tasks within 128 tries against 56% for the post-trained version. Their Sharpening Tax came out positive in 36 of 42 model and benchmark pairs, meaning coverage shrank.
Before picking a post-trained model for an agent that can retry, measure how many tasks it ever solves, not just first-try accuracy.
