⭐ Get the Agent OS + join 3,400+ founders inside the AI Profit Boardroom → Join AIPB

GoldieBench Blog · 7 min read

Hermes vs Copilot: Which AI Models To Run With Each Tool, What The GoldieBench Scores Say, And How To Pin One Per Job

Copilot hands you a menu. Hermes hands you the whole kitchen. The bench helps with both, if you read it the right way.

A dark navy leaderboard with gold bars, split into two panels, one showing a code branch icon and the other a chat bubble and clock, with model cards slotting into each

Hermes vs Copilot is also a model question: GitHub Copilot gives you a model menu inside GitHub's tools, while Hermes Agent lets you plug in almost any provider for each chat, cron job and subagent.

This page pairs both tools with models scored on GoldieBench, using only numbers I read on live goldiebench.com pages on 7 November 2026.

GoldieBench doesn't score Hermes or Copilot as products.

It scores the models you'd run inside them.

If you want my team to choose a model stack with you, get your Backlink Gameplan, or try Polar Rank free for 3 days.

Why the model question splits in two

In Copilot, GitHub decides which models you can pick.

When you assign an issue to the Copilot cloud agent (formerly the Copilot coding agent), the dialog has a model dropdown and, where the model supports it, a reasoning level (GitHub docs).

In Hermes, you decide.

hermes model switches between providers such as Nous Portal, OpenRouter, OpenAI or your own endpoint with no code changes (Hermes providers).

So for Copilot the bench helps you choose from a menu, and for Hermes it helps you build the menu.

What a GoldieBench score actually measures

The methodology page sets the rules: one fixed prompt per task, the same string for every model, one shot, no retries and no help.

The output must be a single self-contained HTML file that runs in a browser.

Julian scores each result from 0 to 10, and a model's leaderboard rank is its average across the tasks it was scored on.

That's a test of one-shot building.

It maps closely to a Copilot task that should end in working code, and loosely to a Hermes cron job that summarises a chat.

The scores, as read on goldiebench.com

ModelAverageScored tasksPage read
Fusion8.59/1047/models/fusion
Claude Opus 58.27/1050/models/opus5
Hermes MoA8.17/1047/models/moa
GPT-5.6 Sol8.16/1050/models/gpt56
Claude Fable 58.10/1047/models/fable-5
Kimi K37.89/1050/models/kimik3
GLM-5.27.77/1047/models/glm
Claude Sonnet 57.01/1047/models/sonnet5

Every row here rests on 47 or 50 scored tasks, so the averages compare like with like.

Medals follow the same scores: per task, the top result takes gold, the next silver and the third bronze, which is why a model can hold several golds and still sit mid-table.

Gaps of a few hundredths, like GPT-5.6 Sol against Claude Fable 5, are too small to decide a pairing on their own.

Fusion is listed as a multi-model panel of Fable 5 and GPT-5.5 through OpenRouter, not a single model.

The full list sits on the all-models page, and members of the AI Profit Boardroom can propose models for it.

Picking a model inside Copilot

GoldieBench has no entry for "Copilot", so read the bench through the dropdown.

Open the model menu in the cloud agent dialog or your editor, then look up each name on the all-models page.

If a model in your menu has a GoldieBench page, its average tells you how it did on one-shot builds.

If it has no page, the bench has nothing to say about it, and I won't guess.

For repo work, the one-shot score is a useful first filter, because a Copilot task is judged on whether the code works when it comes back.

Picking a model inside Hermes

Hermes isn't tied to one vendor's menu: whatever your connected provider offers can sit behind it, and Fusion and Hermes MoA both run through OpenRouter according to their GoldieBench pages.

For a main chat model that also writes code, Claude Opus 5 has the highest single-model average in this table at 8.27/10.

GPT-5.6 Sol at 8.16/10 and Claude Fable 5 at 8.10/10 sit close behind.

Kimi K3 at 7.89/10 and GLM-5.2 at 7.77/10 trail the leaders by a few tenths, and the GLM-5.2 page lists open weights and a 1M-token context.

The Hermes default isn't on the bench

If you never pick a model, Hermes v0.23.0 moved its silent default from GLM-5.2 to GLM-5.3 (release notes).

GoldieBench scores GLM-5.2, not GLM-5.3.

So the 7.77/10 belongs to the old default, and there's no bench number yet for the new one.

Run hermes model and choose on purpose instead of inheriting a default you haven't measured.

Routing Hermes through your Copilot plan

Nous Research's github-copilot labs plugin lets Hermes use a Copilot subscription as its provider.

Two commands set it up: hermes plugins install github-copilot, then hermes auth add copilot for the device-code login (plugin README).

Until you choose Copilot, the Hermes model picker shows a curated Copilot list.

That means the same "look it up on the bench" rule applies: match each Copilot model name to a GoldieBench page before you pin it.

The plugin is built by Nous but labelled unsupported, so keep a second provider set as a fallback.

One model per job with cron pins

Hermes cron lets you pin a model per job, set a fleet default with cron.model, or fall back to your main model (cron docs).

JobWhere it runsMy pick from the bench
Fix a bug from an issueCopilot cloud agentThe highest-scoring model in your Copilot menu that has a GoldieBench page
Main Hermes chat that also codesHermesClaude Opus 5 (8.27/10)
A high-stakes one-shot buildHermesFusion (8.59/10) or Hermes MoA (8.17/10) through OpenRouter
Morning digest of Copilot sessionsHermes cronA pinned lighter model; the bench measures builds, not summaries

The Hermes MoA page puts its default panel at Opus 4.8 plus GPT-5.5, merged by an Opus 4.8 aggregator, and notes that each build takes longer than a single model's call.

Hermes itself has a /moa command that runs one prompt through its default Mixture of Agents preset.

Two small-business pairings

A web agency with client repos

  1. The developers assign small fixes to Copilot and pick the best-scoring model their menu offers.
  2. Hermes runs its main chat on Claude Opus 5 for code questions from the team's Slack.
  3. A pinned cron job lists finished Copilot sessions each morning with gh agent-task list.

A bakery with an online order form

  1. The order form's code sits in a small GitHub repo that a freelancer maintains with Copilot.
  2. The owner never opens Copilot and uses Hermes on Telegram for supplier research and email drafts, marked draft only.
  3. That chat runs on GPT-5.6 Sol or Claude Fable 5, both within a few hundredths of 8.1 on the bench.

Comparing Hermes with Cursor instead? My Hermes vs Cursor model guide runs the same exercise for that pair.

Verdict: how to use the bench for each tool

  • Copilot: use GoldieBench as a filter on GitHub's menu, and only for names that have a page.
  • Hermes: use it to choose the main model, then pin lighter models on cron jobs where one-shot build quality doesn't matter.
  • Both: recheck the pages when they update, because averages move as new tasks are scored.

Read the model pages, pin one model per job, and get your Backlink Gameplan if you want help choosing models for hermes vs copilot, or try Polar Rank free for 3 days.

FAQ

Does GoldieBench score GitHub Copilot or Hermes?

No. It scores models on one-shot builds, so use it to choose the models you run inside each tool.

Which model does Hermes use if I never pick one?

Hermes v0.23.0 made GLM-5.3 the silent default. GoldieBench scores GLM-5.2 at 7.77/10 but has no GLM-5.3 page yet.

What is the top single model in this table?

Claude Opus 5 at 8.27/10 across 50 scored tasks. Fusion scores higher at 8.59/10, but it is a multi-model panel.

Can Hermes use the models in my Copilot plan?

Yes, through the github-copilot labs plugin from Nous Research, which adds copilot and copilot-acp providers.

The same stack Julian uses

Run this stack yourself.

Every demo on this bench was built inside the Agent Operating System — one prompt, one shot, single HTML file out. The Agent OS, the prompts, the templates, the weekly walkthroughs and 3,400+ founders shipping with it every day all live inside the AI Profit Boardroom.

3,400+founders
258documented wins
38countries
4coaching calls a week