Deepnote Launches Agent Initiative Benchmark OpenAI Takes Top Two at 17x Lower Cost

Deepnote founder Jakub Jurovych has released the Agent Initiative Benchmark, placing 12 frontier models into a simulated enterprise Slack group as AI colleagues to test their ability to judge when to intervene and when to stay silent. OpenAI's GPT-6 Astra and GPT-6.1 Sol took the top two spots, with Sol costing only $0.11 per task, while third-ranked Anthropic Fable 5.2 cost $1.87 per task and scored lower, translating to roughly a 16x difference in monthly bills for a 70-person company.

The benchmark builds a synthetic enterprise Slack with six years of detailed history, requiring agents to autonomously decide whether to speak, which services to call, and whom to @. Two LLM judges score responses against reference answers, and both the dataset and code are open-sourced. Extreme comparisons show Fable 5.1 at $3.02 per task versus GPT-6 Luna at $0.007—a 430x gap—with only a 1.1-point score difference, highlighting massive cost-performance disparities.

This benchmark shifts evaluation from answer accuracy to judgment in proactive collaboration, providing a reproducible quantitative basis for enterprise agent selection. OpenAI's dual advantage in cost and performance could accelerate the deployment of AI colleagues in real workflows and prompt the industry to re-examine agent economics.