GoldieBench deep dive
Grok's full benchmark breakdown.
Snappy + real-time — the X-native model. Every number below is either one of my own judged one-shot builds - playable on this site - or an outside result with its source linked. Nothing display-only, nothing vibes.
01 · The headline numbers
02 · Every benchmark, bar by bar
All 43 scored tasks, best first. Hover for the judge's comment; click through for every model's build on that task. Or overlay other models in the interactive graphs.
03 · Where it wins - with the judge's own words
“Twilight Vale — 3D open-world RPG with hand-crafted village, NPCs, combat, day/night, weather, inventory. 38KB — densest build of the bench, edges out Fusion's 32KB.”
“Open-ended 'make a game' — Grok shipped a juicy 28KB build with score HUD, lives, sound, polish.”
“A genuinely premium keynote page: clean nav, a gradient headline, dual buttons, tasteful type. From one sentence. Grok Build's best work of the lot.”
“35KB top-down RPG with tilemap, walkable terrain, NPCs, combat, HP/MP UI, inventory. Beats Fusion's lighter 26KB attempt on density.”
“Web-OS desktop with wallpaper, dock, draggable resizable windows for Notes/Paint/Terminal/Calculator. 33KB — beats Fusion's 24KB attempt on density.”
04 · Where it struggles - quoted, not hidden
Boards that hide the weak rows are brochures. These are Grok's lowest scored builds, verdicts unedited:
“Classic matrix rain. Smaller build (6KB) — straightforward implementation, on brief.”
“Aurora ribbons over mountain ridge with stars. Simpler build (9KB) — lighter on detail than Fusion.”
“Tron procedural terrain flythrough with glowing grid mountains. Drag to steer. 18KB.”
05 · Category breakdown vs the whole field
Games (19 tasks)
Others (0 tasks)
Pages (3 tasks)
Sims (12 tasks)
Visuals (9 tasks)
06 · Outside signals - every row sourced
My bench measures one thing: judged one-shot builds. These outside rows measure other things - kept separate, never blended into the GoldieBench average, every value linked to where it comes from.
Vendor-published numbers
| Benchmark | Result | Source |
|---|---|---|
| SWE-bench Pro | 64.7% | x.ai |
| Terminal-Bench 2.1 | 83.3% | x.ai |
| DeepSWE 1.0 | 62.0% | x.ai |
| SWE Marathon (pass@1) | 29.0% - ahead of Opus 4.8 (26.0%) and Fable max (24.0%) | x.ai |
| Token efficiency | 15,954 avg output tokens per SWE-bench Pro task vs 67,020 for Opus 4.8 max - a 4.2x gap | x.ai |
Independent evaluations
| Benchmark | Result | Source |
|---|---|---|
| BenchLM weighted board | 6th overall (76.97) - 6th coding (71.25) - 12th agentic (62.09), Supported evidence | benchlm.ai |
07 · Head-to-head records
08 · Methodology + honest limits
Every GoldieBench score is a one-shot, single-file build from an identical prompt - no retries, no hand-fixing - rendered for real, screenshotted, and scored 0-10 by one judge model on one rubric across the whole field. Failures score as failures. What this bench does NOT measure: multi-turn agent work, long-context recall, or API latency - that is what the sourced outside rows are for. Full method: /methodology.
Frequently asked questions
01How does Grok perform on GoldieBench?
Grok averages 8.09/10 across 43 scored one-shot build tasks, ranking #6 of 17 ranked frontier models, with 5 task golds, 1 silvers and 5 bronzes. Every score is a real judged build you can open and play on this site.
02What is Grok best at?
Its strongest scored build is Twilightvale at 9.5/10. By category it averages Game 8.3, Page 8.3, Sim 8.0, Visual 7.7 on the bench.
03How much does Grok cost?
Bundled with X (Twitter) Premium subscription — no per-token bill for end users, no individual API pricing for the chat product.
Source ledger
- 01Official vendor site: grok.comgrok.com
- 02SWE-bench Prox.ai
- 03BenchLM weighted boardbenchlm.ai
- 04Julian's guide: /grok-buildagentos.guide
Every model's breakdown
Run this stack yourself.
Every demo on this bench was built inside the Agent Operating System — one prompt, one shot, single HTML file out. The Agent OS, the prompts, the templates, the weekly walkthroughs and 4,000+ founders shipping with it every day all live inside the AI Profit Boardroom.