⭐ Get the Agent OS + join 3,400+ founders inside the AI Profit Boardroom → Join AIPB

GoldieBench Blog · 6 min read

Hermes WhatsApp Group Chat + Which AI Models To Run Behind It, With Real GoldieBench Scores

Access is set by the group policy, not the model. Quality is set by the model. Here is how to match each to a staff group, a project group and a scheduled-post group.

Illustrated leaderboard bars beside a stream of chat bubbles entering a gold gate on a dark navy background

The model you run behind a Hermes WhatsApp group chat bot decides how good its answers are, but it doesn't decide who it answers: that is set by the group policy, the sender allowlist and mention mode, all configured outside the model.

This page pairs the two. First the group settings that apply to any model, then which models on the GoldieBench board make sense for group work, with every score quoted from the live GoldieBench models page as read on 13 November 2026.

If you want my team to help you choose, book a free SEO strategy session.

What The Model Changes In A Group Bot, And What It Doesn't

Swap the model and you change answer quality, speed and how much context it can hold.

Swap the model and you don't change access. The Hermes WhatsApp docs put group intake behind the group policy, which defaults to pairing and forwards nothing from groups.

So set the access rules once, then pick the model. You choose your provider and model with hermes model.

What GoldieBench Actually Measures

The GoldieBench methodology gives every model the same fixed prompts, one shot each, and each result must be a single self-contained HTML file that runs in a browser.

Scores are Julian's 0 to 10 ratings, and the leaderboard ranks models by their average across scored tasks.

That is a test of one-shot building, not of chat replies in a WhatsApp group. Read the scores as a signal of overall capability, not as a group-chat benchmark.

The Top Of The Board Right Now

ModelAverageTasksSource
Fusion8.5947/models/
Claude Opus 58.2750/models/
Hermes MoA8.1747/models/moa
GPT-5.6 Sol8.1650/models/
Claude Fable 58.1047/models/
Grok8.0947/models/
Kimi K37.8950/models/kimik3
Gemini 3.6 Flash7.0850/models/gemini36

Averages and task counts are exactly as listed on those pages.

A Fast Lane For Mention Replies

Most group traffic, once mention mode is on, is short: a question, a quick lookup, a reply to the bot's last message.

For that, speed matters more than peak score. The Gemini 3.6 Flash page lists it at 7.08 average over 50 tasks and describes "Fast one-shot builds", with best-for notes that include high-volume agentic work.

It sits lower on the board than the flagships, so treat it as the everyday lane and keep a stronger model for harder asks.

A Heavy Lane For Long Group Tasks

Some group requests are bigger: summarise a week of messages, plan a job, draft a document from a quoted file.

Claude Opus 5 sits at 8.27 over 50 tasks on the board, the highest single-vendor model listed.

The Kimi K3 page lists 7.89 over 50 tasks and a context of 1,048,576 tokens, and says it is used for long unattended agent runs inside the Agent OS.

Either suits a group where a few people send long, tool-heavy tasks.

The Claude Opus 5.5 page is a useful contrast: it lists a 1,000,000-token context but a 7.57 average over 50 tasks, below Opus 5 on this board. Newer doesn't automatically mean higher here.

Hermes MoA In A Group: When A Panel Makes Sense

The Hermes MoA page lists 8.17 over 47 tasks: a panel of frontier models whose drafts are merged by an aggregator.

The same page notes the trade-off: latency of "~110–140s per single-file build" versus a solo model's one call.

In a busy group, that wait is long for a quick question. Save a panel for scheduled jobs, such as a nightly digest, where nobody is waiting on a reply.

What The Model Sees From A Group Message

The bridge hands the model more than plain text, which is worth knowing when you compare models.

  • Quoted media: when someone quotes a photo or file, Hermes passes that attachment to the model along with the question, so a model that handles images well pays off in groups where people share photos.
  • Batched bursts: a rapid run of messages from one chat arrives as one combined request after a short quiet period, 0.3 seconds by default.
  • Polls: when the model asks a multiple-choice clarify question, Hermes sends it as a native WhatsApp poll. Approval prompts never become polls.
  • Long answers: replies are split at 4,096 characters per chunk, so a verbose model sends several messages into the group.

That last point favours a model that answers tightly. In a group, three chunks of text is a lot to scroll past.

Shared Sessions And Context Size

Context size matters more when a group shares one conversation.

The configuration docs set group_sessions_per_user: true by default, so each person gets their own session. Set it to false and the group shares one session, along with its token costs and context growth.

If you share a session, a model with a large context window is the safer choice. If you keep per-person sessions, the fast lane usually covers it.

Wire The Group Settings First

  • Pair a dedicated number in bot mode with hermes whatsapp.
  • Set WHATSAPP_GROUP_POLICY=allowlist and list group JIDs in WHATSAPP_GROUP_ALLOWED_USERS, per the environment variables reference.
  • Set WHATSAPP_ALLOWED_USERS so only approved senders get through.
  • Turn on WHATSAPP_REQUIRE_MENTION=true to keep model calls to real requests.

Mention mode does double duty here. Fewer triggers means fewer model calls, which keeps a stronger model practical to run in a busy group.

Use the Baileys bridge for this. The Cloud API adapter handles direct messages only in v1.

A Pairing Plan For Three Kinds Of Group

  • A busy staff group with quick questions: per-person sessions, mention mode on, a fast model.
  • A small project group sharing one thread: shared session, a large-context model such as Kimi K3 or Claude Opus 5.
  • A group that only receives scheduled posts: a cron job to the group JID, where a slower panel like Hermes MoA has time to work.

One number runs one WhatsApp session, and the Hermes FAQ says you can't bind different profiles to different chats on it. To use different models for different groups on one number, switch models within a single profile rather than splitting profiles.

Reading The Scores Without Over-Reading Them

A gap of a few hundredths, such as 8.17 against 8.16, is close to a tie on a one-shot build board.

Task counts matter too. Grok 4.7, for example, is listed on the board with 20 tasks, while most of the models above have 47 or 50, so its average rests on fewer builds.

And the board changes as new models are benched, so check the live page before you switch, not a copy of it.

The bigger signal for a group bot is fit: speed for quick replies, context for shared threads, patience for scheduled work.

Running the bot in DMs as well? Our Hermes Agent WhatsApp model guide covers one-to-one chats.

To learn how to wire these models into your own Agent OS, the AI Profit Boardroom has the trainings.

Also On Our Network

Set the group rules first, then pick a model from the board, and if you want help choosing, book a free strategy session or join the AI Profit Boardroom to run models behind your Hermes WhatsApp group chat.

FAQ

Does the model decide who the bot answers in a group?

No. Group policy, the sender allowlist and mention mode decide that. The model only shapes the answer.

Which model scores highest on GoldieBench right now?

Fusion, at 8.59 over 47 tasks on goldiebench.com/models/ as read on 13 November 2026. Claude Opus 5 is next at 8.27 over 50 tasks.

Is Hermes MoA a good fit for a busy group?

Its GoldieBench page notes about 110 to 140 seconds per single-file build, so it suits scheduled posts better than quick replies.

Can I run a different model for each WhatsApp group?

Not as separate profiles on one number, because one number is one session. Use one profile and switch models, or give each profile its own number.

The same stack Julian uses

Run this stack yourself.

Every demo on this bench was built inside the Agent Operating System — one prompt, one shot, single HTML file out. The Agent OS, the prompts, the templates, the weekly walkthroughs and 3,400+ founders shipping with it every day all live inside the AI Profit Boardroom.

3,400+founders
258documented wins
38countries
4coaching calls a week