⭐ Get the Agent OS + join 3,400+ founders inside the AI Profit Boardroom → Join AIPB

GoldieBench Blog · 5 min read

Hermes Agent Gemini 4: Setup Steps, Model Specs and What Google's Cards Report

The setup for Gemini 4 in Hermes Agent, with model specs and Google's published results.

Three faceted gold crystals of different sizes on a minimalist grid over dark navy

Hermes Agent Gemini 4 setup is straightforward, but GoldieBench has not yet benched any Gemini 4 model, so this guide pairs the setup steps with what Google's own model cards report and how to pick a Gemini 4 model for build work in Hermes.

Book a free SEO strategy session →

To see how we test tools for SEO work, book a free SEO strategy session.

Setup facts come from the Hermes Gemini guide; model data from Google's Hydrogen model card and model pages.

Setup

# 1. add your key (GEMINI_API_KEY also works)
echo "GOOGLE_API_KEY=your-key-here" >> ~/.hermes/.env

# 2. pick the provider and model
hermes model
#   More providers... -> Google AI Studio -> gemini-4-hydrogen

# 3. check it, then chat
hermes doctor
hermes chat

Use the native endpoint, https://generativelanguage.googleapis.com/v1beta; long decode does not run on the OpenAI-compatible endpoint or OpenRouter.

Gemini 4 models in Hermes

ModelWindowDefault output per requestThinking defaultTier
gemini-4-hydrogen1,048,576 unified65,536highFree and paid
gemini-4-argon1,048,576 unified65,536mediumPaid
gemini-4-carbon1,048,576 unified65,536mediumPaid only

All three share a March 2026 knowledge cutoff, per Google's model cards, and none supports explicit caching.

What Google's cards report

Benchmark (Google card)Gemini 4 HydrogenClaude Sonnet 5.5
Agents' Last Exam v1.171.660.4
Chartography66.661.6
OSWorld-2.1 offline77.683.9
SWE-sweep13.53.1

These are Google's figures, not ours.

They suggest Hydrogen is strong on agent tasks and charts but behind Sonnet 5.5 on computer use.

Picking a Gemini 4 model for builds

For one-shot builds inside Hermes, start on Hydrogen at high effort, which is Google's default for it, and compare with Carbon on your hardest briefs.

Our guide to the best LLMs for Hermes Agent covers the models we have already benched.

Effort settings that affect builds

Hermes effortArgon / CarbonHydrogen
lowlowlow
mediummediummedium
highhighhigh
xhighhigh + 262,144-token long decodexhigh + 262,144-token long decode
maxhigh + long decode to window minus inputxhigh + long decode to window minus input

Book a free SEO strategy session →

The mapping is from the Hermes guide.

For a one-shot build that fits in a single HTML file, long decode is rarely needed; reasoning quality decides the score.

More rows from Google's cards

BenchmarkHydrogenClaude Sonnet 5.5
HLE-Diamond58.853.3
Terminal-Bench-Science 0.151.459.9
BenchmarkCarbonArgonClaude Opus 5.5
FrontierSWE 261.255.062.3
Terminal-Bench 4.065.257.159.6
OSWorld-2.1 offline81.574.787.2

From Google's Hydrogen and Carbon model cards; these are vendor figures, not GoldieBench runs.

Release timeline

Google's changelog lists Argon as generally available on 13 October 2026, Carbon on 10 November and Hydrogen on 12 November.

Google's model cards describe Carbon as based on Argon and Hydrogen as adapted from Carbon.

All three arrived after our current Opus 5.5 and Fable 5 runs, so they are next in the queue.

How we will run Gemini 4

  • Through the native AI Studio provider, the same route the Hermes guide recommends.
  • Effort pinned per run, since Hydrogen defaults to high thinking and Argon and Carbon to medium.
  • The same fixed prompts as every model on the board, one shot each, with playable demos published.

Long decode in practice

Without long decode, a single Gemini request stops at 65,536 output tokens when streaming, thinking included; Google's long decode docs explain how continuation tokens extend that.

Hermes joins the hops into one reply, but each hop re-reads the prompt and the output so far, so billed input grows quickly.

For builds, that cost rarely pays off; for very long documents it can.

Fallback for bench-style runs

Hermes's fallback providers let you put a benched model behind Gemini 4, so a rate limit or safety block does not end a run.

For fair benchmarking we turn fallbacks off, so every score reflects one model; for production agents, turn them on.

Hermes is open source on GitHub, so anyone can reproduce the setup we will use.

Verdict

Set up the native provider, default to Hydrogen and add a benched model as a fallback until Gemini 4 joins our board.

For a Hermes Agent OS with routing built in, see the AI Profit Boardroom.

Book a free SEO strategy session →

Go native, start on Hydrogen and Hermes Agent Gemini 4 is ready for build work; for data-led SEO, book a free SEO strategy session.

FAQ

Has GoldieBench tested Gemini 4 in Hermes?

Not yet. No Gemini 4 model has a GoldieBench score so far.

Which Gemini 4 model is best for builds in Hermes?

Start with Hydrogen at high effort and compare with Carbon on the hardest briefs.

Do all Gemini 4 models have the same context window?

Yes. Hydrogen, Argon and Carbon share a 1,048,576-token unified window.

The same stack Julian uses

Run this stack yourself.

Every demo on this bench was built inside the Agent Operating System — one prompt, one shot, single HTML file out. The Agent OS, the prompts, the templates, the weekly walkthroughs and 3,400+ founders shipping with it every day all live inside the AI Profit Boardroom.

3,400+founders
258documented wins
38countries
4coaching calls a week