GoldieBench Blog · 7 min read
Hermes vs Copilot: Which AI Models To Run With Each Tool, What The GoldieBench Scores Say, And How To Pin One Per Job
Copilot hands you a menu. Hermes hands you the whole kitchen. The bench helps with both, if you read it the right way.

Hermes vs Copilot is also a model question: GitHub Copilot gives you a model menu inside GitHub's tools, while Hermes Agent lets you plug in almost any provider for each chat, cron job and subagent.
This page pairs both tools with models scored on GoldieBench, using only numbers I read on live goldiebench.com pages on 7 November 2026.
GoldieBench doesn't score Hermes or Copilot as products.
It scores the models you'd run inside them.
If you want my team to choose a model stack with you, get your Backlink Gameplan, or try Polar Rank free for 3 days.
Why the model question splits in two
In Copilot, GitHub decides which models you can pick.
When you assign an issue to the Copilot cloud agent (formerly the Copilot coding agent), the dialog has a model dropdown and, where the model supports it, a reasoning level (GitHub docs).
In Hermes, you decide.
hermes model switches between providers such as Nous Portal, OpenRouter, OpenAI or your own endpoint with no code changes (Hermes providers).
So for Copilot the bench helps you choose from a menu, and for Hermes it helps you build the menu.
What a GoldieBench score actually measures
The methodology page sets the rules: one fixed prompt per task, the same string for every model, one shot, no retries and no help.
The output must be a single self-contained HTML file that runs in a browser.
Julian scores each result from 0 to 10, and a model's leaderboard rank is its average across the tasks it was scored on.
That's a test of one-shot building.
It maps closely to a Copilot task that should end in working code, and loosely to a Hermes cron job that summarises a chat.
The scores, as read on goldiebench.com
| Model | Average | Scored tasks | Page read |
|---|---|---|---|
| Fusion | 8.59/10 | 47 | /models/fusion |
| Claude Opus 5 | 8.27/10 | 50 | /models/opus5 |
| Hermes MoA | 8.17/10 | 47 | /models/moa |
| GPT-5.6 Sol | 8.16/10 | 50 | /models/gpt56 |
| Claude Fable 5 | 8.10/10 | 47 | /models/fable-5 |
| Kimi K3 | 7.89/10 | 50 | /models/kimik3 |
| GLM-5.2 | 7.77/10 | 47 | /models/glm |
| Claude Sonnet 5 | 7.01/10 | 47 | /models/sonnet5 |
Every row here rests on 47 or 50 scored tasks, so the averages compare like with like.
Medals follow the same scores: per task, the top result takes gold, the next silver and the third bronze, which is why a model can hold several golds and still sit mid-table.
Gaps of a few hundredths, like GPT-5.6 Sol against Claude Fable 5, are too small to decide a pairing on their own.
Fusion is listed as a multi-model panel of Fable 5 and GPT-5.5 through OpenRouter, not a single model.
The full list sits on the all-models page, and members of the AI Profit Boardroom can propose models for it.
Picking a model inside Copilot
GoldieBench has no entry for "Copilot", so read the bench through the dropdown.
Open the model menu in the cloud agent dialog or your editor, then look up each name on the all-models page.
If a model in your menu has a GoldieBench page, its average tells you how it did on one-shot builds.
If it has no page, the bench has nothing to say about it, and I won't guess.
For repo work, the one-shot score is a useful first filter, because a Copilot task is judged on whether the code works when it comes back.
Picking a model inside Hermes
Hermes isn't tied to one vendor's menu: whatever your connected provider offers can sit behind it, and Fusion and Hermes MoA both run through OpenRouter according to their GoldieBench pages.
For a main chat model that also writes code, Claude Opus 5 has the highest single-model average in this table at 8.27/10.
GPT-5.6 Sol at 8.16/10 and Claude Fable 5 at 8.10/10 sit close behind.
Kimi K3 at 7.89/10 and GLM-5.2 at 7.77/10 trail the leaders by a few tenths, and the GLM-5.2 page lists open weights and a 1M-token context.
The Hermes default isn't on the bench
If you never pick a model, Hermes v0.23.0 moved its silent default from GLM-5.2 to GLM-5.3 (release notes).
GoldieBench scores GLM-5.2, not GLM-5.3.
So the 7.77/10 belongs to the old default, and there's no bench number yet for the new one.
Run hermes model and choose on purpose instead of inheriting a default you haven't measured.
Routing Hermes through your Copilot plan
Nous Research's github-copilot labs plugin lets Hermes use a Copilot subscription as its provider.
Two commands set it up: hermes plugins install github-copilot, then hermes auth add copilot for the device-code login (plugin README).
Until you choose Copilot, the Hermes model picker shows a curated Copilot list.
That means the same "look it up on the bench" rule applies: match each Copilot model name to a GoldieBench page before you pin it.
The plugin is built by Nous but labelled unsupported, so keep a second provider set as a fallback.
One model per job with cron pins
Hermes cron lets you pin a model per job, set a fleet default with cron.model, or fall back to your main model (cron docs).
| Job | Where it runs | My pick from the bench |
|---|---|---|
| Fix a bug from an issue | Copilot cloud agent | The highest-scoring model in your Copilot menu that has a GoldieBench page |
| Main Hermes chat that also codes | Hermes | Claude Opus 5 (8.27/10) |
| A high-stakes one-shot build | Hermes | Fusion (8.59/10) or Hermes MoA (8.17/10) through OpenRouter |
| Morning digest of Copilot sessions | Hermes cron | A pinned lighter model; the bench measures builds, not summaries |
The Hermes MoA page puts its default panel at Opus 4.8 plus GPT-5.5, merged by an Opus 4.8 aggregator, and notes that each build takes longer than a single model's call.
Hermes itself has a /moa command that runs one prompt through its default Mixture of Agents preset.
Two small-business pairings
A web agency with client repos
- The developers assign small fixes to Copilot and pick the best-scoring model their menu offers.
- Hermes runs its main chat on Claude Opus 5 for code questions from the team's Slack.
- A pinned cron job lists finished Copilot sessions each morning with
gh agent-task list.
A bakery with an online order form
- The order form's code sits in a small GitHub repo that a freelancer maintains with Copilot.
- The owner never opens Copilot and uses Hermes on Telegram for supplier research and email drafts, marked draft only.
- That chat runs on GPT-5.6 Sol or Claude Fable 5, both within a few hundredths of 8.1 on the bench.
Comparing Hermes with Cursor instead? My Hermes vs Cursor model guide runs the same exercise for that pair.
Verdict: how to use the bench for each tool
- Copilot: use GoldieBench as a filter on GitHub's menu, and only for names that have a page.
- Hermes: use it to choose the main model, then pin lighter models on cron jobs where one-shot build quality doesn't matter.
- Both: recheck the pages when they update, because averages move as new tasks are scored.
Read the model pages, pin one model per job, and get your Backlink Gameplan if you want help choosing models for hermes vs copilot, or try Polar Rank free for 3 days.
FAQ
Does GoldieBench score GitHub Copilot or Hermes?
No. It scores models on one-shot builds, so use it to choose the models you run inside each tool.
Which model does Hermes use if I never pick one?
Hermes v0.23.0 made GLM-5.3 the silent default. GoldieBench scores GLM-5.2 at 7.77/10 but has no GLM-5.3 page yet.
What is the top single model in this table?
Claude Opus 5 at 8.27/10 across 50 scored tasks. Fusion scores higher at 8.59/10, but it is a multi-model panel.
Can Hermes use the models in my Copilot plan?
Yes, through the github-copilot labs plugin from Nous Research, which adds copilot and copilot-acp providers.
- 📺 Video notes + links to the tools
- 🆓 Get a FREE AI course + community + 1,000 AI agents
- 🎥 Learn how I make these videos


