⭐ Get the Agent OS + join 3,400+ founders inside the AI Profit Boardroom → Join AIPB

GoldieBench Blog · 6 min read

Hermes Agent Fallback Model: How To Set Up Backup Models, What Triggers Them And How To Benchmark Your Own Chain

Set the chain, learn the triggers, then measure every backup with the live readiness check before an outage measures it for you.

Illustrated hero: a primary agent node handing off to a gold backup node beside a small leaderboard of response bars on a dark navy background

The hermes agent fallback model setup is a short list of backup provider:model pairs in your config that Hermes switches to when the main model errors, and the part most people skip is measuring those backups before an outage forces the test.

This page covers the setup, the exact triggers, the separate fallback for side tasks, and how to treat the new readiness check like a small benchmark of your own chain.

Setup details come from the official Fallback Providers documentation and the Hermes CLI reference.

Near the end there's one short section on choosing the backup model itself.

If you'd rather have my team look at your setup, book a free strategy session.

Fallback In One Paragraph

When the primary fails, Hermes resolves the backup's credentials, builds a new client, swaps model and provider in place, picks up the backup's own reasoning effort, resets its retry counter and continues the same conversation. Nothing in the history is lost. On the next user message it tries the primary again.

That's the primary model layer. Same-provider key rotation, through credential pools, runs before it, and side tasks have a ladder of their own.

Set Up The Chain

Two ways in. The manager:

hermes fallback add
hermes fallback list

Or the file, ~/.hermes/config.yaml:

fallback_providers:
  - provider: nous
    model: nous-hermes-3
  - provider: openrouter
    model: anthropic/claude-sonnet-4

Entries run top to bottom. Each needs a provider and a model; a half-written entry is dropped.

Gemini entries accept gemini, google, google-gemini or google-ai-studio as the provider name, per the docs.

The Triggers, Grouped By Timing

ErrorRetries first?Then
429 rate limitYesSwitch, primary benched until its named reset
500, 502, 503YesSwitch
401, 403NoSwitch immediately
404NoSwitch immediately
Repeated empty or malformed repliesYesSwitch

A provider that gives no reset time gets an exponential backoff. If you'd prefer to wait out a reset that's only moments away, set fallback.min_switch_reset_seconds; the default 0 always switches.

One Turn At A Time, And What It Costs

A switch covers the current turn only. The following user message is sent to the primary first, and inside one turn Hermes falls back no more than once.

If the backup errors too, ordinary retry handling runs and then an error is shown, which keeps a pair of failing providers from looping.

The exception is a primary that has declared a reset time still in the future. Hermes stays on the backup until that moment instead of making a request it expects to fail.

The price of every switch is the prompt cache. The incoming model has no cached copy of the conversation, so it reads the full history, and so does the primary on its return.

That's why a backup that answers reliably beats a backup that's merely available: fewer failed attempts means fewer full re-reads.

Side Tasks Fall Back Separately

Vision, compression, MCP helpers, approval checks, title generation, review and the triage specifier each sit under auxiliary.<task>.

Left on auto, a side task tries the main provider and model, then its own fallback_chain, then the main chain, and skips with a warning if all fail.

Pinned to a named provider, it only moves on for capacity errors: a 402, a daily-quota 429 or a dead connection. Per-entry timeout values let a slower backup finish instead of dying on the primary's clock.

Compression degrades gently: with no model, it drops middle turns instead of failing the session.

Benchmark Your Own Chain With hermes fallback check

The readiness check appears in the v0.23.2 release notes of 16 November 2026. Think of it as a tiny benchmark run on your own backups.

hermes fallback check --live --json

For every entry the JSON reports fields including provider, model, auth_ready, inference_ready, latency_ms, model_returned and status.

Three of those are worth tracking over time:

  • status tells you pass or fail: ready, or a reason such as auth, model_mismatch or live_rate_limit.
  • latency_ms shows how quickly each backup answered its one short request.
  • model_returned shows what the provider actually served, which is how a model_mismatch is caught.

Each entry is tested on its own provider in parallel, never answered by another, so a fast first entry can't hide a dead second one.

The command exits 1 on any failure, which makes a nightly run easy to alert on.

Where The Chain Applies

  • CLI sessions, including one-shot hermes -z runs, and the startup case where the primary's login fails first.
  • The messaging gateway and the Desktop and TUI apps.
  • Unpinned cron jobs. Pinned ones fail hard instead.
  • Subagents, through delegation.fallback_providers or by inheriting when unpinned.

Edit the chain while a chat is open and the change applies from its next turn.

Local Primary, Hosted Backup

The providers page documents a local Ollama model as the main custom provider. Add a hosted entry to fallback_providers and the cloud model covers the times your machine can't answer.

Hermes needs 64,000 tokens of context or more for agent use, so set the local model's context length to match.

Write local backups as custom with a base_url: an open GitHub issue (#74280) reports that provider: ollama entries never activate.

The Community Option

model-fallback-heuristic, a community plugin by tomkubik at version 0.2.0, fills fallback_providers automatically by ranking models from public catalogs on things like parameter count, context window and tool support. Nous Research didn't make it.

It rewrites the chain at session start and after API errors, keeps your hand-written entries, and leaves the primary alone by default. It fits free-model setups where IDs change often.

Results That Should Worry You

  • ready (static) only. You never ran --live, so nothing proves the backup answers.
  • auth on an entry you set months ago. Keys and logins expire; replace it now.
  • model_mismatch. The provider is serving something other than the model you named.
  • Every entry on one provider. The check can pass while your chain still shares a single point of failure.

Choosing The Backup Model

This is the one place model choice matters, and the bar is lower than for a primary.

  • Different provider from the primary, so they don't fail together.
  • Tool calling, since Hermes is running agent steps, not just chat.
  • Enough context to take over a long conversation mid-turn.
  • Answers the live check reliably, with a latency_ms you can live with.

For ranked candidates, our best free AI model for Hermes Agent guide compares options, and the GoldieBench leaderboard shows current scores.

My Verdict On Setting Up A Hermes Fallback

Add one backup on a different provider, run hermes fallback check --live --json, and keep the output so you can compare runs.

Schedule that check, leave cron jobs unpinned, and add side-task chains only where one task needs a specific model.

Want to swap notes with people running the same setup? The AI Profit Boardroom has weekly calls for exactly that.

Run the live check tonight, keep the JSON, and if you want help tuning the chain, join the AI Profit Boardroom or book a free strategy session for your hermes agent fallback model.

FAQ

What does hermes fallback check --live actually measure?

It sends one short request to each fallback entry on that entry's own provider and reports whether it answered, how long it took and which model replied.

Can I compare backup models with the check output?

Yes, roughly. The JSON includes latency_ms and status per entry, so repeated runs show which backup answers reliably and quickly.

Does a fallback change the model's reasoning effort?

Yes. Hermes looks up the backup model's own effort setting instead of reusing the primary's.

Which errors skip the retries and switch straight away?

Auth failures (401 and 403) and not-found errors (404). Rate limits and server errors are retried first.

What should a good backup model have?

A different provider from your primary, tool calling support, enough context for long sessions and a clean pass on the live check.

Is model-fallback-heuristic an official Hermes plugin?

No. tomkubik, a community developer, publishes it, and the Hermes Marketplace lists it.

The same stack Julian uses

Run this stack yourself.

Every demo on this bench was built inside the Agent Operating System — one prompt, one shot, single HTML file out. The Agent OS, the prompts, the templates, the weekly walkthroughs and 3,400+ founders shipping with it every day all live inside the AI Profit Boardroom.

3,400+founders
258documented wins
38countries
4coaching calls a week