GoldieBench Blog · 5 min read
Hermes Agent Gemini 4: Setup Steps, Model Specs and What Google's Cards Report
The setup for Gemini 4 in Hermes Agent, with model specs and Google's published results.

Hermes Agent Gemini 4 setup is straightforward, but GoldieBench has not yet benched any Gemini 4 model, so this guide pairs the setup steps with what Google's own model cards report and how to pick a Gemini 4 model for build work in Hermes.
Book a free SEO strategy session →
To see how we test tools for SEO work, book a free SEO strategy session.
Setup facts come from the Hermes Gemini guide; model data from Google's Hydrogen model card and model pages.
Setup
# 1. add your key (GEMINI_API_KEY also works)
echo "GOOGLE_API_KEY=your-key-here" >> ~/.hermes/.env
# 2. pick the provider and model
hermes model
# More providers... -> Google AI Studio -> gemini-4-hydrogen
# 3. check it, then chat
hermes doctor
hermes chat
Use the native endpoint, https://generativelanguage.googleapis.com/v1beta; long decode does not run on the OpenAI-compatible endpoint or OpenRouter.
Gemini 4 models in Hermes
| Model | Window | Default output per request | Thinking default | Tier |
|---|---|---|---|---|
| gemini-4-hydrogen | 1,048,576 unified | 65,536 | high | Free and paid |
| gemini-4-argon | 1,048,576 unified | 65,536 | medium | Paid |
| gemini-4-carbon | 1,048,576 unified | 65,536 | medium | Paid only |
All three share a March 2026 knowledge cutoff, per Google's model cards, and none supports explicit caching.
What Google's cards report
| Benchmark (Google card) | Gemini 4 Hydrogen | Claude Sonnet 5.5 |
|---|---|---|
| Agents' Last Exam v1.1 | 71.6 | 60.4 |
| Chartography | 66.6 | 61.6 |
| OSWorld-2.1 offline | 77.6 | 83.9 |
| SWE-sweep | 13.5 | 3.1 |
These are Google's figures, not ours.
They suggest Hydrogen is strong on agent tasks and charts but behind Sonnet 5.5 on computer use.
Picking a Gemini 4 model for builds
For one-shot builds inside Hermes, start on Hydrogen at high effort, which is Google's default for it, and compare with Carbon on your hardest briefs.
Our guide to the best LLMs for Hermes Agent covers the models we have already benched.
Effort settings that affect builds
| Hermes effort | Argon / Carbon | Hydrogen |
|---|---|---|
| low | low | low |
| medium | medium | medium |
| high | high | high |
| xhigh | high + 262,144-token long decode | xhigh + 262,144-token long decode |
| max | high + long decode to window minus input | xhigh + long decode to window minus input |
Book a free SEO strategy session →
The mapping is from the Hermes guide.
For a one-shot build that fits in a single HTML file, long decode is rarely needed; reasoning quality decides the score.
More rows from Google's cards
| Benchmark | Hydrogen | Claude Sonnet 5.5 |
|---|---|---|
| HLE-Diamond | 58.8 | 53.3 |
| Terminal-Bench-Science 0.1 | 51.4 | 59.9 |
| Benchmark | Carbon | Argon | Claude Opus 5.5 |
|---|---|---|---|
| FrontierSWE 2 | 61.2 | 55.0 | 62.3 |
| Terminal-Bench 4.0 | 65.2 | 57.1 | 59.6 |
| OSWorld-2.1 offline | 81.5 | 74.7 | 87.2 |
From Google's Hydrogen and Carbon model cards; these are vendor figures, not GoldieBench runs.
Release timeline
Google's changelog lists Argon as generally available on 13 October 2026, Carbon on 10 November and Hydrogen on 12 November.
Google's model cards describe Carbon as based on Argon and Hydrogen as adapted from Carbon.
All three arrived after our current Opus 5.5 and Fable 5 runs, so they are next in the queue.
How we will run Gemini 4
- Through the native AI Studio provider, the same route the Hermes guide recommends.
- Effort pinned per run, since Hydrogen defaults to high thinking and Argon and Carbon to medium.
- The same fixed prompts as every model on the board, one shot each, with playable demos published.
Long decode in practice
Without long decode, a single Gemini request stops at 65,536 output tokens when streaming, thinking included; Google's long decode docs explain how continuation tokens extend that.
Hermes joins the hops into one reply, but each hop re-reads the prompt and the output so far, so billed input grows quickly.
For builds, that cost rarely pays off; for very long documents it can.
Fallback for bench-style runs
Hermes's fallback providers let you put a benched model behind Gemini 4, so a rate limit or safety block does not end a run.
For fair benchmarking we turn fallbacks off, so every score reflects one model; for production agents, turn them on.
Hermes is open source on GitHub, so anyone can reproduce the setup we will use.
Verdict
Set up the native provider, default to Hydrogen and add a benched model as a fallback until Gemini 4 joins our board.
For a Hermes Agent OS with routing built in, see the AI Profit Boardroom.
Book a free SEO strategy session →
Go native, start on Hydrogen and Hermes Agent Gemini 4 is ready for build work; for data-led SEO, book a free SEO strategy session.
FAQ
Has GoldieBench tested Gemini 4 in Hermes?
Not yet. No Gemini 4 model has a GoldieBench score so far.
Which Gemini 4 model is best for builds in Hermes?
Start with Hydrogen at high effort and compare with Carbon on the hardest briefs.
Do all Gemini 4 models have the same context window?
Yes. Hydrogen, Argon and Carbon share a 1,048,576-token unified window.
- 📺 Video notes + links to the tools
- 🆓 Get a FREE AI course + community + 1,000 AI agents
- 🎥 Learn how I make these videos


