Independent AI model evaluation and analysis firm providing third-party comparisons of model performance, pricing and speed.
Artificial Analysis released Endpoint Accuracy Index v1.1, tightening confidence intervals and reducing grader error on its long-context AA-LCR-25 benchmark while adding coverage for GLM-5.3 Flash and Muse Spark Open. Of 103 serverless endpoints benchmarked, 24 score statistically below the self-hosted reference, quantifying accuracy trade-offs from quantization, custom kernels and inference-stack tuning.
Google and evaluation firm Artificial Analysis released MKA, a benchmark of 200 multimodal knowledge-work tasks spanning 10 industries that mirror 3-8 hours of expert human effort. Gemini 4 Carbon placed second while costing roughly one-sixth of the top model per task, and the benchmark is now open to developers via Artificial Analysis.
Google DeepMind, in partnership with Artificial Analysis, has launched the Multimodal Knowledge Agent (MKA) benchmark to evaluate agentic knowledge work over complex visual files. It comprises 200 private real-world tasks across 10 industries, requiring agents to produce deliverables such as slide decks, spreadsheets, and reports from dense visual inputs like architectural drawings, engineering schematics, and clinical scans, emphasizing realistic visual complexity and domain specificity.
Google released Gemini Omni Flash 1.2 on November 5, taking second place on Artificial Analysis' AA-Video-Editing v1.1 leaderboard while climbing from #12 to #6 in text-to-video and #17 to #14 in image-to-video. The model supports text-to-video, image-to-video and video editing with native audio and clips up to 15 seconds, priced at $6.26 per minute of 1080p video, 31% below its predecessor, and will replace version 1.1 on the Gemini API on December 3.
AI evaluation firm Artificial Analysis held a San Francisco launch event for Intelligence Index v5, outlining where AI benchmarking is heading next. The index is a widely referenced gauge of model intelligence, but the public post offers only event highlights and imagery, with no details yet on methodology or results.
Google DeepMind and Artificial Analysis have partnered to launch MKA (Multimodal Knowledge Agent), a benchmark evaluating how well agents handle visually complex tasks such as cross-referencing blueprints and reading scientific charts, which existing evaluations do not cover. Gemini 4 Carbon ranks second on the benchmark at 6x lower average cost per task than the top model.
Google DeepMind, in partnership with Artificial Analysis, launched the Multimodal Knowledge Agent (MKA) evaluation, a benchmark of 200 private real-world visual tasks spanning 10 industries. Agents must interpret dense visual files such as architectural drawings, engineering schematics and clinical scans, then produce deliverables including slide decks, spreadsheets and reports. Scoring uses expert rubrics split into Climb and Regression criteria, averaging about 70 weighted items per task, with Claude Sonnet 5.5 leading in several domains.
Keenable Search 1.0 ranks first in pro mode with 83 and second in realtime mode with 82 on the Artificial Analysis Search Index, both ahead of Exa and Perplexity at 80. Priced at $4 per 1k searches, it costs about $0.040 per task, over a third below the board median, while completing tasks in roughly 15 seconds for 2nd and 3rd place on speed. Both modes improved about 15 points from the 67 scored in the August benchmark.
Artificial Analysis built VIALS-AA on Handshake AI's VIALS benchmark to test visual reasoning over life-science artifacts such as gel blots, plasmid maps and phylogenies. Claude Fable 5.2 leads at 67%, ahead of Gemini 4 Carbon (64%) and GPT-6 Astra (63%). The more notable signal is tool-use strategy: Fable averages roughly 37 tool calls per task with heavy batching, versus about 8 for Astra, suggesting reasoning models are shifting toward multi-step tool orchestration rather than single-shot answers.
Artificial Analysis reports that Claude Fable 5.2 (Max) now ranks first on AA-3D, its benchmark for interactive 3D generation, scoring 1271 Elo, 67 points ahead of Claude Sonnet 5.5 and Opus 5.5, with a confidence interval that overlaps no other model. It gains 132 points over its predecessor Fable 5.1 and beats the top OpenAI and Google entries, GPT-6 Astra (1168) and Gemini 4 Argon (1147).