⭐ Get the Agent OS + join 3,400+ founders inside the AI Profit Boardroom → Join AIPB

GoldieBench Blog · 6 min read

Hermes Agent Home Assistant: How to Connect It, What the Four Tools Do and How to Test the Link

A setup guide with a test plan, so you can prove the connection works the same way every time.

Illustrated smart home devices beside a benchmark checklist with pass marks on a dark navy background with gold accents

Hermes Agent Home Assistant support comes from one official plugin, and a short test plan tells you whether your link is solid.

Set it up in five steps, then run the tests in order before you let the agent control anything.

Book a free SEO strategy session →

For help turning AI into search traffic, book a free SEO strategy session with Goldie Agency.

The steps follow the official Home Assistant page in the Hermes docs.

Setup in five steps

  1. hermes plugins install homeassistant, repeated with -p for each extra profile.
  2. Create a Long-Lived Access Token in your Home Assistant profile.
  3. Put HASS_TOKEN in ~/.hermes/.env, plus HASS_URL if HA is not at http://homeassistant.local:8123.
  4. Add a watch list in config.yaml: watch_domains, watch_entities or watch_all.
  5. Run hermes gateway.

The four tools in one table

ToolInputReturns
ha_list_entitiesOptional domain or areaEntity IDs, states, friendly names
ha_get_stateentity_idState, attributes, last changed and updated
ha_list_servicesOptional domainAvailable actions and their parameters
ha_call_servicedomain, service, entity_id, dataRuns the action

Source for each tool lives in the plugin repo.

Test 1: reachability

From the Hermes host, call the Home Assistant API with your token as a bearer header against /api/.

A working link returns the API running message.

A 401 means the token is wrong or short-lived.

Test 2: read path

Ask the agent to list every light in one room.

Pass if it returns entity IDs and states that match the HA dashboard.

Test 3: event path

Open a door sensor you are watching.

Pass if the agent receives a triggered event, and a second one inside 30 seconds is held back by the cooldown.

Test 4: write path

Ask for one harmless change, such as a lamp turning on.

Pass if the lamp changes and the agent reports the call it made.

Test 5: blocked domains

Ask the agent to run a shell_command service.

Pass if it returns an error, because the plugin blocks shell_command, command_line, python_script, pyscript, hassio and rest_command.

Book a free SEO strategy session →

Test 6: delivery

Create a scheduled job that delivers to homeassistant:mobile_app_my_phone.

Pass if the note lands on your phone at the set time; the cron docs cover the schedule syntax.

Scoring your setup

ResultMeaningNext step
All six passReady for routinesAdd domains one at a time
Test 3 failsWatch list empty or wrongFix filters, restart gateway
Test 1 or 2 failsToken or URL problemNew long-lived token, check scheme and port
Test 6 failsNotify target wrongCheck the target name in HA

Test 7: reconnect behaviour

Restart Home Assistant while the gateway is running.

Pass if the gateway reconnects on its own; the plugin backs off from 5 to 10 to 30 to 60 seconds between attempts.

Fail if you have to restart Hermes by hand, which usually points to a URL or network problem rather than the plugin.

Test 8: profile isolation

If you run more than one Hermes profile, ask the second profile to list your lights.

Pass if it cannot, because the plugin and HASS_TOKEN live only in the profile you installed them in.

Test 9: upgrade check

After each hermes update, rerun tests 2, 3 and 5.

The upgrade path keeps your token, URL, platform keys and cron syntax, but a quick rerun proves it on your own setup.

Event format reference

DomainHow the event reads
climateHVAC mode change with current and target values
sensorOld reading to new reading
binary_sensorTriggered or cleared
light, switch, fanTurned on or turned off
alarm_control_panelAlarm state change, old to new
anything elseOld state to new state

Use this table to write test expectations, so a pass or fail is never a judgement call.

Why a fixed test plan beats a quick try

A quick "turn on the lamp" proves one path on one day.

Book a free SEO strategy session →

The plugin has four separate paths: reads, writes, live events and delivery, and each can break for a different reason.

Reads break on tokens and URLs, events break on watch lists, writes break on service names, and delivery breaks on notify targets.

Testing each path on its own tells you which part failed instead of leaving you to guess.

It also gives you a baseline to compare against after every update or Home Assistant change.

Settings that change the results

SettingEffect on tests
watch_domainsDecides which events test 3 can see
ignore_entitiesRemoves noisy sensors before any other filter
cooldown_secondsChanges how often repeat events arrive, default 30
HASS_HOME_CHANNELSets the target for the bare deliver form in test 6
display.platforms.homeassistantAdds or removes progress chatter in notifications

Record these values with each run so results stay comparable.

What a clean run looks like

A clean run is boring: every test passes, the reports arrive on time and nothing changes that you did not ask for.

Keep that baseline, and any surprise later stands out at once.

How often to rerun the plan

Run the full plan once after setup, then tests 2, 3 and 5 after every Hermes update or Home Assistant upgrade.

Run test 7 whenever your network changes, such as a new router or a moved server.

That rhythm catches problems before they show up as a missed alert.

Common failure patterns and what they mean

A pass on test 2 and a fail on test 3 almost always means an empty or mistyped watch list.

A fail on test 1 with a 401 means the token is short-lived or pasted with quotes around it.

A pass on test 4 with the wrong device changing means the entity ID in the prompt was wrong, not the plugin.

A fail on test 6 with everything else passing points at the notify target name.

Keeping notes on each run

Write down the date, the Hermes version and the result of every test.

When something breaks after an update, the notes show exactly which path changed.

That habit is the difference between a guess and a benchmark.

Does the model matter here?

The plugin does the device work, so most current models handle the four tool calls without trouble.

GoldieBench tests models on building tasks rather than smart home control, so no score here would tell you anything useful for this job.

Pick a model your Hermes setup already runs well and judge it on the six tests above.

Next builds

Add webhook routes when another service should trigger the agent.

Read the Hermes repo for the gateway internals.

The AI Profit Boardroom covers the next agent builds step by step.

Book a free SEO strategy session →

Run all six tests once, save the results, and repeat them after every Hermes update.

Then book a free SEO strategy session with Goldie Agency, and keep this test plan for your Hermes Agent Home Assistant setup.

FAQ

How do I test my Hermes Agent Home Assistant link?

Check reachability with a token request to /api/, then test reads, events, a harmless write, a blocked domain and a scheduled delivery.

What does the cooldown do?

It sets a minimum of 30 seconds by default between forwarded events for the same entity, so noisy sensors do not flood the agent.

Which model should run the Home Assistant tools?

Any model your Hermes setup already handles well; the plugin does the device work and the calls are simple.

Can I use it in more than one Hermes profile?

Yes. Install the plugin in each profile with hermes -p <profile> plugins install homeassistant.

The same stack Julian uses

Run this stack yourself.

Every demo on this bench was built inside the Agent Operating System — one prompt, one shot, single HTML file out. The Agent OS, the prompts, the templates, the weekly walkthroughs and 3,400+ founders shipping with it every day all live inside the AI Profit Boardroom.

3,400+founders
258documented wins
38countries
4coaching calls a week