GoldieBench deep dive
Fugu Ultra's full benchmark breakdown.
Sakana's multi-agent answer to Fusion — frontier ensemble without single-vendor risk. Every number below is either one of my own judged one-shot builds - playable on this site - or an outside result with its source linked. Nothing display-only, nothing vibes.
01 · The headline numbers
02 · Every benchmark, bar by bar
All 42 scored tasks, best first. Hover for the judge's comment; click through for every model's build on that task. Or overlay other models in the interactive graphs.
03 · Where it wins - with the judge's own words
“Ultra v2 (gap-fill) — click-to-launch fireworks. Smoke-test PASS with 26.3% pixel diff — among the most reactive builds.”
“Ultra v2 — juicy browser game. Smoke-test PASS with 24.7% pixel diff — one of the most reactive builds on the bench.”
“Sakana Fugu Ultra shipped a 32KB Apple-keynote landing — bigger than Fusion's 20KB attempt at the same prompt. Animated mesh gradient, multi-section, polished. $0.32 vs Fusion's $1.30 for the same output — 4× cheaper, denser result.”
“Ultra v2 (gap-fill) — 61.5KB Nordic dungeon crawler with bloom + boss room. Smoke-test PASS with 22.8% pixel diff — highly reactive.”
“Fugu Ultra v2 rebuild — 55.7KB solar system, the densest solar attempt on the bench. Full </html>, animation loop, Saturn rings, drag-to-orbit + scroll-to-zoom. Smoke-test PASS (3.6% pixel diff after drag, zero console errors). The panel ensemble produces a markedly richer build than Mini's 12KB single-model version.”
04 · Where it struggles - quoted, not hidden
Boards that hide the weak rows are brochures. These are Fugu Ultra's lowest scored builds, verdicts unedited:
“Ultra v2 — billiards. Smoke-test MAYBE (0.0% diff) — cue needs click-drag aim that generic input didn't replicate.”
“Ultra v2 — arcade shooter. Smoke-test MAYBE (0.0% diff) — may need a specific start interaction; flagged for manual verification.”
“Ultra v2 — fluid sim. Smoke-test MAYBE (0.0% diff) — needs click-drag injection which generic input didn't replicate.”
“Ultra v2 — wormhole tunnel. Smoke-test MAYBE (0.3% diff) — animates slowly; hold-space accel didn't show big change in the short test.”
05 · Category breakdown vs the whole field
Games (19 tasks)
Others (0 tasks)
Pages (2 tasks)
Sims (12 tasks)
Visuals (9 tasks)
06 · Outside signals - every row sourced
My bench measures one thing: judged one-shot builds. These outside rows measure other things - kept separate, never blended into the GoldieBench average, every value linked to where it comes from.
From the source guides
| Measure | Result | Source guide |
|---|---|---|
| SWE Bench Pro | 73.7 | /sakana.ai/fugu |
| GPQA-Diamond | 95.5 | /sakana.ai/fugu |
| MRCRv2 | 93.6 | /sakana.ai/fugu |
07 · Head-to-head records
08 · Methodology + honest limits
Every GoldieBench score is a one-shot, single-file build from an identical prompt - no retries, no hand-fixing - rendered for real, screenshotted, and scored 0-10 by one judge model on one rubric across the whole field. Failures score as failures. What this bench does NOT measure: multi-turn agent work, long-context recall, or API latency - that is what the sourced outside rows are for. Full method: /methodology.
Frequently asked questions
01How does Fugu Ultra perform on GoldieBench?
Fugu Ultra averages 7.94/10 across 42 scored one-shot build tasks, ranking #8 of 17 ranked frontier models, with 5 task golds, 2 silvers and 7 bronzes. Every score is a real judged build you can open and play on this site.
02What is Fugu Ultra best at?
Its strongest scored build is Fireworks at 9.0/10. By category it averages Game 7.7, Page 8.0, Sim 8.1, Visual 8.2 on the bench.
03How much does Fugu Ultra cost?
Sakana's multi-agent orchestration: a single API call internally dispatches to multiple frontier models and synthesises the answer. Subscription plans run $20-$200/mo (Standard / Pro / Max); PAYG is $5/M input + $30/M output for Fugu Ultra. Direct competitor to OpenRouter Fusion's panel approach.
Source ledger
- 01Official vendor site: sakana.aisakana.ai
- 02SWE Bench Proagentos.guide
- 03Julian's guide: /sakana-fugu-vs-fusionagentos.guide
Every model's breakdown
Run this stack yourself.
Every demo on this bench was built inside the Agent Operating System — one prompt, one shot, single HTML file out. The Agent OS, the prompts, the templates, the weekly walkthroughs and 4,000+ founders shipping with it every day all live inside the AI Profit Boardroom.