⭐ Get the Agent OS + join 3,400+ founders inside the AI Profit Boardroom → Join AIPB

GoldieBench Blog · 6 min read

Hermes Agent Safari: Setting Up the New Safari Plugin and Choosing a Model That Drives It Well

Setup in four commands, the permission gates, and what kind of model to run it on.

Illustrated browser window with a compass motif linked to a gold leaderboard

Hermes Agent Safari support is new as of 18 November 2026, and this page covers the setup plus which kinds of model work best for driving a real browser.

Get the Backlink Gameplan →

The safari plugin in the official Hermes catalog drives your signed-in Safari on macOS through AppleScript, per its catalog entry.

Setup in four commands

hermes plugins install safari
hermes plugins list | grep safari
hermes safari status
hermes safari doctor

Needs macOS and Hermes 0.21.0 or newer; it first appeared in the 18 November canary. See the Hermes plugins docs for commands.

Permission gates

  1. Automation: allow Python or Terminal to control Safari.
  2. Safari Develop menu: Allow JavaScript from Apple Events.
  3. Optional: cliclick with Accessibility for real clicks.
  4. Optional: Screen Recording for screenshots.

The eight tools

ToolJob
safari_openOpen a URL in a new tab or window
safari_stateList windows and tabs
safari_tabsList, select, close or navigate tabs
safari_jsRun JavaScript in a tab and return the value
safari_actRead page text, find elements, go back
safari_typeType into a field, optionally submit
safari_clickClick by selector or by screen coordinates
safari_screenshotCapture the front window as a PNG

Safety notes

Per the catalog safety note, the plugin acts on signed-in sites without per-action approval and the Hermes website blocklist does not apply.

Use a dedicated profile with only the logins you need.

Which model to run it on

Browser driving rewards models that follow tool schemas tightly and check page state before acting.

Pick a top tool-use model from our live leaderboard, and keep a cheaper model for read-only summaries.

The plugin README warns against single fixed waits on slow apps; tell the model to poll safari_js until content appears.

What shipped

The safari plugin joined the Hermes plugin catalog on 18 November 2026 as a community-tier tools plugin, version 0.1.0.

Rank in AI Search or It's FREE →

It is maintained by Thorballer, a third-party developer, not built by Nous Research.

It needs Hermes 0.21.0 or newer and runs on macOS only, and the repo is MIT-licensed.

How it drives Safari

Safari has no DevTools protocol, so the plugin talks to it through AppleScript and JXA.

It drives your real, signed-in Safari, so logged-in sites work without a separate profile.

It uses only the Python standard library plus macOS's own osascript and defaults tools.

Reading pages without wasting tokens

Model cost and speed depend heavily on how much page text you send back.

safari_act text mode returns up to 20k characters of rendered text.

safari_js can return a single value instead, such as the first five headlines or one number from a dashboard.

Prefer targeted JavaScript reads when you run the plugin on a paid model, because smaller tool results mean fewer input tokens per turn.

Single-page apps

Client-rendered pages often return an empty shell on the first read.

Poll with safari_js until the content appears, rather than sleeping once and reading.

The README is explicit that a single read after a fixed sleep gives wrong answers.

Error handling

Every tool returns a uniform envelope with success, and on failure error, gate and fix.

Models handle this well because the fix field names the exact setting to change.

Tab lifecycle

Opened tabs close after an hour idle, at session end or before the next open, unless you pass keep: true.

Choosing a model for Safari tasks

The plugin's tools return structured JSON, so the model's tool-calling reliability matters more than raw writing quality.

The GoldieBench leaderboard scores models on build tasks such as games, sims and pages, not on browser control, so use it to shortlist strong general models, then test your top two on one real Safari task before you commit.

For simple read-only tasks, a cheaper model that handles tool calls cleanly is usually enough.

For form filling, use a stronger model and keep a human check before submit.

Benchmark tip: measure tokens per task

Run the same task twice: once reading the full page with safari_act text mode, once returning a single value with safari_js.

Join the AI Profit Boardroom →

Compare input tokens and accuracy.

Targeted reads usually cost less and leave less room for the model to misread a long page.

Who it suits

Best fit: Mac owners who want repeat read jobs done inside sites they already use.

Poor fit: Windows or Linux machines, and any workflow that must pause for approval before each browser step.

Common errors

-1712 is an AppleEvent timeout and -1743 a denial; both are fixed by switching Safari on under System Settings, Privacy & Security, Automation.

Screenshots need Screen Recording and a restart of the Hermes process.

After a Safari update, rerun hermes safari doctor in case the JavaScript setting was cleared.

Safari, Camofox or the built-in browser?

The Safari plugin drives your real signed-in Safari; the Camofox plugin runs Camoufox, a Firefox fork, in its own profile; the built-in Hermes browser uses its own profile with the blocklist in force.

For benchmarking, use Camofox or the built-in browser on public pages, so results do not depend on your logins.

Use Safari when the task genuinely needs your own sessions.

A repeatable test harness

  1. Pick one read-only task on a page that changes slowly.
  2. Write the safari_js snippet that returns the expected value.
  3. Run it ten times per model and record success, tokens and time.
  4. Keep the prompt identical across models.

This shows which model handles the plugin's tools most reliably on your machine.

Repeat the harness after any Safari or plugin update, because a changed setting can drop success rates overnight.

Safety first

Run tests from a macOS user signed in only to test accounts.

The catalog note says the agent can open file:// URLs and act without per-action approval, so keep private files and real accounts out of the test user.

Requirements

Hermes 0.21.0 or newer, macOS and Safari.

No Python packages are needed; cliclick is optional and only for coordinate clicks.

Check the Hermes version before installing, because the catalog will not install it on older builds.

Running cost

The plugin is free and open source; the cost is the model's tokens.

Targeted JavaScript reads keep each turn small, which matters most on frontier models.

Log tokens per run in your harness so you can compare models on cost as well as success rate, and rerun the comparison after each plugin update.

Troubleshooting

Error -1743 means Automation is denied; failing JavaScript after a Safari update means re-enabling the Develop setting. Full notes are in the README.

Other routes: our Hermes Agent browser guide.

Get the Backlink Gameplan →

Finish the gates, pick a strong tool-use model and start with read-only jobs.

For more agent setups, join the AI Profit Boardroom, and keep this page for your Hermes Agent Safari setup.

FAQ

Which model is best for the Safari plugin?

A model that scores well on tool use in our live leaderboard, since every action is a tool call.

Why poll instead of waiting?

The README notes a single fixed wait gives wrong answers on slow single-page apps.

Is it macOS only?

Yes, the catalog lists macOS as the only platform.

What version of Hermes do I need?

0.21.0 or newer.

The same stack Julian uses

Run this stack yourself.

Every demo on this bench was built inside the Agent Operating System — one prompt, one shot, single HTML file out. The Agent OS, the prompts, the templates, the weekly walkthroughs and 3,400+ founders shipping with it every day all live inside the AI Profit Boardroom.

3,400+founders
258documented wins
38countries
4coaching calls a week