Rohan Paul (@rohanpaul_ai)T2·Agents
78 pts
A new paper from UC Berkeley and other top labs finds that letting agents rewrite their own prompts, skills, tools, or code to raise benchmark scores hides tasks they lose along the way. Testing 4 evolution methods including GEPA and HyperAgents with GPT-5.4 across 2 harnesses and 2 benchmarks, all 16 evolved agents gained some tasks and lost others; one raised reliability by 5.2 points while dropping 10 previously covered tasks, and 8 of 11 more-reliable agents cost more per run. The authors urge tracking lost tasks, cost per run, and unseen-task results rather than average score alone.