⭐ Get the Agent OS + join 3,400+ founders inside the AI Profit Boardroom → Join AIPB

GoldieBench Blog · 6 min read

Hermes vs Gemini Spark: Which AI Models Hermes And Tasks in Gemini Run, With Real GoldieBench Scores

Gemini Spark is now Tasks in Gemini, and Google picks its model. Hermes lets you pick yours. Here is how the scores on GoldieBench line up.

Illustrated leaderboard with glowing bars comparing AI models, split between a self-hosted agent panel and a cloud task panel, on dark navy with gold accents

Hermes vs Gemini Spark splits the model question in two: Google picks the model behind its agent, while Hermes Agent runs whichever model you connect.

Gemini Spark (now called Tasks in Gemini) is Google's renamed task mode, per its help page, and its menu lets you switch between Google's own models.

This page matches both tools with the models scored on GoldieBench, using only numbers I read on live goldiebench.com pages on 6 November 2026.

GoldieBench doesn't score either agent as a product.

It scores the models they run.

If you want my team to help choose a model stack, get your Backlink Gameplan, or try Polar Rank free for 3 days.

Why the two agents treat models so differently

Tasks in Gemini is a Google service.

Its help page says you can change the Gemini model or thinking level from the menu at the top of a task, and that is the whole choice.

Hermes is open source and provider-neutral.

You run hermes model and choose a route from the providers page: Nous Portal, OpenRouter, Anthropic, OpenAI, a Gemini API key, Vertex AI, or a model on your own hardware.

You can also pin a different model to each cron job, so a nightly summary and a weekly build don't have to share one.

Which model Google says powers the agent

Google has named a different Flash model at different times.

DateWhat Google saidSource
19 May 2026Spark runs on Gemini 3.5 with the Antigravity harness; the Gemini 3.5 post says 3.5 FlashGemini app launch post
29 Jul 2026India rollout post: "Driven by Gemini 3.6 Flash"Google India blog
13 Aug 2026Spark starts using Gemini 3.7 FlashGemini 3.7 Flash post

Google hasn't posted a model change since the rename, so I won't claim which Flash model sits behind Tasks in Gemini today.

What's clear is that it's a Google Flash-tier model, and you don't get to swap in another vendor.

What a GoldieBench score measures

The methodology page gives every model the same fixed prompt, one shot, with no retries and no help.

Each result must be one self-contained HTML file that runs in a browser.

Julian scores every build from 0 to 10, and the board ranks models by their average across scored tasks.

So the score tells you how well a model builds a page, game or simulation in one go.

It says less about how well a model files your email, which is most of what Tasks in Gemini does.

Gemini 3.6 Flash on the bench

Gemini 3.6 Flash is the one Spark-era Google model I found on GoldieBench.

MeasureGemini 3.6 FlashPage read
GoldieBench average7.08/10/models/gemini36
Scored tasks50/models/gemini36
Board rank#19 of 26 ranked frontier models/benchmarks/gemini36
Landing Page task8.6/models/gemini36
Matrix Visual task8.7, gold/models/gemini36

The model page says it was benched through the native Gemini API on launch day, not through the Gemini app.

Gemini 3.7 Flash, the model Google named in August, isn't on the board yet.

Models you can run in Hermes, with their scores

These are the top averages on the GoldieBench models page as I read it, plus the Hermes Mixture of Agents setup.

Model or setupGoldieBench averageScored tasks
Fusion8.59/1047
Claude Opus 58.27/1050
Hermes MoA8.17/1047
GPT-5.6 Sol8.16/1050
Claude Fable 58.10/1047
Kimi K37.89/1050
GLM-5.27.77/1047
Gemini 3.6 Flash7.08/1050

The Hermes MoA page describes a panel of Claude Opus 4.8 and GPT-5.5 whose drafts are merged by Opus 4.8, run from the Mixture tab in the Agent OS.

It is a setup you build inside Hermes, not something Tasks in Gemini can do.

A note on Hermes defaults

The v0.23.0 release notes move the silent default model, used when you never pick one, from GLM-5.2 to GLM-5.3.

GLM-5.2 scores 7.77 on the bench; GLM-5.3 has no GoldieBench score yet.

The same notes list Gemini 4 Argon among newly supported models, and it isn't on the board either.

So pick your model on purpose rather than leaving the default.

Pairing models with real business jobs

A bakery's new landing page. This is the closest job to what GoldieBench tests.

In Hermes, pin a top scorer such as Claude Opus 5 or the MoA setup to the build job, then review the page.

In Tasks in Gemini, ask for an HTML file; Google lists .html among the file types it can create, and Gemini 3.6 Flash scored 8.6 on the bench's Landing Page task.

An estate agency's morning email digest. GoldieBench doesn't measure inbox triage.

In Hermes, use a cron job with a lighter model pinned through the per-job model setting and deliver the digest to Slack.

In Tasks in Gemini, a Gmail monitor or a daily schedule does this inside Google, where the model is chosen for you.

A UK accountant. The model question never comes up for Gemini, because Google doesn't offer tasks in the United Kingdom.

Hermes with a Gemini API key is one way a UK firm can still put Google's models to work inside an agent.

Speed and panels: what the model pages add

The scores aren't the only useful line on these pages.

The Gemini 3.6 Flash page lists fast one-shot builds as a strength, with full 3D games in roughly 60 to 120 seconds of generation.

The Hermes MoA page lists latency as a trade-off: the slowest panel draft plus the merge pass, about 110 to 140 seconds per single-file build.

So a panel buys a higher average at the cost of waiting longer for each result.

For a nightly job nobody watches, that wait doesn't matter much.

Run your own mini test in Hermes

You can copy the bench's method for your own work.

Pick one real job, such as a booking page for a yoga studio, and write one fixed prompt for it.

Run that prompt through two or three models with /model in Hermes, one shot each, and save each HTML file.

Score them 0 to 10 yourself, then pin the winner to the job.

The AI Profit Boardroom has the Agent OS the bench itself runs in.

What the scores don't tell you

GoldieBench measures one-shot builds, not Gmail labelling, calendar clean-ups or form filling.

It also scores models through their APIs, so an app's own wrapper, like Google's task planner, isn't part of the number.

For a different tool pairing on the same site, see Hermes vs n8n with GoldieBench scores.

The model verdict

  • Pick Tasks in Gemini if you're happy with Google's Flash models and want them wired straight into personal Gmail and Calendar.
  • Pick Hermes if you want the higher-scoring models on the board, a per-job model, or Gemini through the API from a country Google's task mode skips.

To test models side by side yourself, the AI Profit Boardroom has the Agent OS that the bench runs in.

Read the model pages, pin one model per job, and get your Backlink Gameplan if you want help choosing models for hermes vs gemini spark, or try Polar Rank free for 3 days.

FAQ

Which model does Tasks in Gemini use?

Google has named Gemini 3.5 Flash, then 3.6 Flash, then 3.7 Flash from 13 August 2026. It has not posted a change since the rename, and the task menu lets you switch between Google models.

What does Gemini 3.6 Flash score on GoldieBench?

Its model page shows a 7.08/10 average across 50 scored tasks, and its benchmark page ranks it 19th of 26 ranked frontier models.

Can Hermes run Gemini models?

Yes, through a Gemini API key or Google Vertex AI. It cannot sign in with a consumer Google AI plan.

Does GoldieBench score Hermes itself?

It scores Hermes MoA, a mixture-of-agents setup built in Hermes, at 8.17/10 across 47 tasks. It does not score the Hermes software as a product.

The same stack Julian uses

Run this stack yourself.

Every demo on this bench was built inside the Agent Operating System — one prompt, one shot, single HTML file out. The Agent OS, the prompts, the templates, the weekly walkthroughs and 3,400+ founders shipping with it every day all live inside the AI Profit Boardroom.

3,400+founders
258documented wins
38countries
4coaching calls a week