Mission Control
benchmarks.arnao.ai

AI Model Benchmarks
& Mission Control

Live tracking of frontier AI models — arena rankings, cost benchmarks, and internal fleet performance data.

Mission Control

🚀 Latest Releases

  1. Sakana Fugu & Fugu Ultra — Jun 22, 2026Multi-agent orchestration model
  2. GLM 5.2 — Jun 16, 2026ZhipuAI’s latest model
  3. Claude Opus 4.8 — Jun 2026Anthropic’s new flagship

🏆 Arena Top 3

  1. Claude Fable 5ELO 1508
  2. Claude Opus 4.6 (Thinking)ELO 1504
  3. Claude Opus 4.7 (Thinking)ELO 1502

⚡ Fleet Winner

🏆 DeepSeek V4 Flash

Best cost/performance ratio in internal fleet tests.

Avg Time: 2.9s  •  Cost: $0.00010 / 4 tasks

LMSYS Chatbot Arena — Top 20

#ModelProviderELO Score

Master Model Catalog

Model Provider Class Arena ELO In $/1M Out $/1M Context Release Status
  • Opus 5 — vetted Jul 28, 2026Released Jul 24, 2026 at unchanged $5/$25 · 1M ctx. Step-change over 4.8 at the SAME price: SWE-bench Verified 96%, SWE-bench Pro 79.2% (vs 4.8's 69.2%), Frontier-Bench 43.3% (2× of 4.8's 21.1%), ARC-AGI-3 1.5%→30.2%, +268 Elo GDPval; approaches Fable 5 on CursorBench/OSWorld at ½–⅓ the cost. Arena ELO pending (not fabricated).
  • CorrectionFixed Opus 4.8 to canonical $5/$25 · 1M (was a stale $15/$75 · 200K). Fable 5 / Opus 4.7 / 4.6 Anthropic prices in this table also look stale — flagged to refresh.
  • SourcesMarkTechPost launch · Vellum · Codersera. Verified 2026-07-28.

Routing Stack Benchmark — Jul 7, 2026

ComboPlanning / ExecutionRefactor (A)Classify (B)Cost/RunLatency
Sonnet Duosonnet-4-6 / sonnet-4-6100/10077/100$0.08438s + 20s
Ollama Stack 🏆qwen3-coder:30b / mistral-small3.2 (local)100/10083/100$0.00078s + 75s
OpenRouter Valueglm-4.6 / gemini-2.5-flash— (credits out)83/100$0.000315s
Gemini Duogemini-2.5-pro / 2.5-flash60/10081/100$0.01730s + 16s
Fable+Gemmaclaude-fable-5 / gemma4 (local)50/10071/100$0.02220s + 48s
  • HeadlineThe all-local Ollama stack tied frontier quality on both tasks at zero marginal cost. Latency is the only tradeoff (2-4x slower).
  • DoctrineCron and batch jobs route local; interactive execution routes to Flash-class; interactive planning routes to Sonnet. Enforced fleet-wide by Ria via routing-rules.json.
  • CaveatFable 5 planning scored 50/100 due to probabilistic safety refusals on code-analysis prompts; strong orchestrator, weak pipeline planner. Benchmarks A/B graded fully automatically (pytest, mypy, AST, gold labels).

Image Model Benchmark — Jul 11, 2026

🏆 Winner: grok-imagine-image-quality (xAI) — nailed text, cat, and puddle count at 6.2s / $0.05
Standard Prompt — identical for every model (stresses exact text, counting, spatial relations, reflections, style) A rain-slicked Tokyo alley at blue hour, seen from a low three-quarter angle. In the foreground, a silver-haired woman in a transparent vinyl raincoat holds a glowing paper lantern shaped like a koi fish; its warm light reflects in exactly three visible puddles. Behind her, a robot barista leans out of a tiny stall serving matcha, with a hand-painted wooden sign that reads exactly 'ARNAO AI LAB' in weathered white letters. A black cat walks along a power line overhead, silhouetted against neon kanji signs. Shot on 35mm film, shallow depth of field focused on the lantern, cinematic teal-and-amber grade, visible rain streaks, steam rising from a manhole.
grok-imagine-image-quality

🏆 grok-imagine-image-quality

xAI · 6.2s · $0.05
Sign correct, cat on wire, ~3 discrete puddles with koi + neon reflections, bonus readable MATCHA neon.
gpt-5.4-image-2

gpt-5.4-image-2

OpenAI via OpenRouter · 180.2s · $0.228
Everything lands — but 30x slower and 4.5x pricier than the winner.
gemini-3-pro-image-preview

gemini-3-pro-image-preview

Google (Nano Banana Pro) · 19.5s · $0.134
Sign + cat correct, superb raincoat and robot barista; puddles read as wet patches.
gemini-3.1-flash-image

gemini-3.1-flash-image

Google via OpenRouter (NB2) · 7.7s · $0.069
Best 35mm street-photo feel; robot barista reduced to an arm.
grok-imagine-image

grok-imagine-image

xAI · 8.0s · $0.02
Cheapest pass: sign, cat, matcha robot all correct; single mega-puddle.
gemini-3.1-flash-lite-image

gemini-3.1-flash-lite-image

Google via OpenRouter · 2.6s · $0.034
Fastest run; sign rendered twice, added an unrequested umbrella.
gpt-image-1

gpt-image-1

OpenAI · 42.0s · $0.167
Sign correct but no cat, weak puddles; secondary MATCHA sign garbled.
gpt-5-image-mini

gpt-5-image-mini

OpenAI via OpenRouter · 46.4s · $0.044
Sign correct; fish reads as glowing object, no cat, no discrete puddles.
gemini-2.5-flash-image

gemini-2.5-flash-image

Google (Nano Banana) · 6.3s · $0.039
Sign reads “ARNAQ AI LAB”; cat perched in a neon sign instead of on a wire.
ModelProviderSign Text3 PuddlesCat on WireAestheticAdherenceLatencyCost
grok-imagine-image-quality 🏆xAI✓✓✓996.2s$0.050
gpt-5.4-image-2OpenAI (OR)✓✓✓99180.2s$0.228
gemini-3-pro-image-previewGoogle✓✗✓9919.5s$0.134
gemini-3.1-flash-imageGoogle (OR)✓✗✓987.7s$0.069
grok-imagine-imagexAI✓✗✓888.0s$0.020
gemini-3.1-flash-lite-imageGoogle (OR)✓✗✓872.6s$0.034
gpt-image-1OpenAI✓✗✗8742.0s$0.167
gpt-5-image-miniOpenAI (OR)✓✗✗8646.4s$0.044
gemini-2.5-flash-imageGoogle✗✗✗876.3s$0.039
dall-e-3OpenAIFAIL — model retired by OpenAI (“does not exist”)
grok-2-imagexAIFAIL — retired, replaced by grok-imagine-image*
moonshot / deepseek—No image-generation API offered
local (ollama / host)—None local — ollama is text-only, no ComfyUI/mflux installed
  • HeadlinexAI’s grok-imagine-image-quality matched gpt-5.4-image-2’s perfect checklist at 1/30th the latency and 1/4.5th the cost. Gia’s image stack now routes winner → Nano Banana Pro → Nano Banana 2.
  • NotableEvery 2026-era model except gemini-2.5-flash-image rendered “ARNAO AI LAB” exactly — text rendering is largely solved. Counting (“exactly three puddles”) remains the hardest test: only 2 of 9 passed.
  • Catalog gapOpenRouter carries no Chinese image-output models today (no Qwen-Image, CogView, Seedream, or Hunyuan-Image) — only Google nano-banana and OpenAI GPT-5-image families.
  • Harnessscripts/benchmarks/image-bench.py — idempotent, per-model reruns via --model, fail-soft error capture. Raw PNGs + results.json in workspace/benchmarks/images/2026-07-11/.

Realtime Voice / Speech-to-Speech Landscape — Jul 26, 2026

🏆 Harness pick: gpt-realtime-2.1-mini (OpenAI) — reasoning + tool-calling in the mini tier, ~$0.04/min real, already wired into the gallery voice guide
Scope — a capability + pricing landscape of NATIVE speech-to-speech (audio-in → audio-out) APIs, web-verified 2026-07-26. Not a self-run latency benchmark; figures are vendor-published. Interaction modes: native S2S (below) vs. composed STT→LLM→TTS (higher latency, "accent-roulette" risk) vs. voice-quality/cloning layer (ElevenLabs/Cartesia — the produced-voice job, not a conversational brain).
ModelProviderMultimodalTool callsBarge-inPrice (audio)ReleasedHarness note
gpt-realtime-2.1-mini 🏆OpenAIaudio+text✓ +reasoning✓~$10/$20 /1M (~$0.04/min)Jul 7 2026Gallery default. WebRTC, ephemeral token via /client_secrets. July drop closed the tool-calling gap.
gpt-realtime-2OpenAIaudio+text✓ GPT-5-class✓$32/$64 /1MMay 7 2026Flagship. HIGH-tier candidate; overkill for card nav.
Gemini 3.1 Flash LiveGoogleaudio+video+img✓ (sync only)✓Flash-tierMar 24 2026Only native model that SEES. The gesture/camera fork — but frames leave device (RAI trade-off).
Nova 2 SonicAmazonaudio✓ async✓$3/$12 /1M (~$0.015/min)Dec 2025~80% cheaper, AWS-native (Bedrock). Documented deployment target, not a running account.
grok-voice-think-fast-1.0xAIaudio✓✓$0.05/minJun 29 2026OpenAI-Realtime compatible — swap base URL to wss://api.x.ai/v1/realtime. Zero-migration A/B.
ElevenAgents / CAI 2.0ElevenLabssee+hear+files✓ +RAG✓platformJan 2026Best voice quality, sub-100ms. Composed platform — reserve for the cloned produced-voice layer.
  • HeadlineThe model is not the bottleneck: gpt-realtime-2.1-mini over WebRTC is native-live-mode-class tech. "Doesn't interact" is tools + turn-config + reliability, not the model — fixed in the gallery voice pass (v75).
  • Gesture fork, resolvedHIGH tier stays OpenAI flagship; gesture stays local MediaPipe (~60fps, zero camera frames leave device). Streaming a webcam to a cloud model to detect a swipe is the less responsible design. Gemini native-vision is shelved as a labeled "frames leave device" experiment, not a tier.
  • AWSNova 2 Sonic is the cheapest + AWS-native path, kept as a documented deployment target in the Mentor/Kiro PRD — the brand story doesn't require running it.
  • SourcesOpenAI voice models · Gemini 3.1 Flash Live · Nova 2 Sonic · Grok Voice. Verified 2026-07-26.

Internal Fleet Benchmark — Jun 21, 2026

ModelAvg TimeCost / 4 TasksPass RateNotes

Research & Deep Dives

  • Sakana FuguAn orchestration model that routes tasks to a team of specialist models. The “satanic fugu” concept in practice.
  • AgentOSA unified command center for managing multiple agents with persistent memory and a shared interface across AI models. A project at agentos.arnao.ai is feasible.
  • Model MonitoringA lightweight cron polls RSS from Hugging Face, arXiv, and top AI labs every 4 hours to surface new releases automatically.