⭐ Get the Agent OS + join 3,400+ founders inside the AI Profit Boardroom → Join AIPB ($49/mo)

GoldieBench Blog · 10 min read

StarNet AI Agent: What It Is, How To Set It Up, And Which Brain Actually Scores Best

StarNet AI agent explained, set up and benchmarked: Grok 4.6 scores 5.95, Grok 4.7 scores 7.15, plus real GoldieBench scores for Claude and Kimi brains.

StarNet AI Agent: What It Is, How To Set It Up, And Which Brain Actually Scores Best — illustrated hero

StarNet AI agent is a free, open-source harness where you create real AI agents and watch them work inside a pixel-art space station on your own computer.

It was made by Andrew Sims, who publishes the code on GitHub as androoAGI.

StarNet itself costs nothing, because the code is released under the MIT License.

The problem it solves is that most agents hide inside a plain chat window, so you can't see what they're doing or who is doing it.

In StarNet, every agent has its own pixel-art look, a place in the station and a role, and the station shows you their real work as it happens.

In this post I'll explain what StarNet is, how it works, how to set it up and how to use every feature, and then I'll be honest about its limits.

What the StarNet AI agent is

StarNet is a local-first desktop harness for building and running teams of AI agents.

The version I tested was v0.13.1, and it runs on your own computer rather than in someone else's cloud.

Your agents live in a pixel-art space station instead of a list of chat threads.

They level up the more you use them, which makes it feel a bit like a Tamagotchi for an AI agent.

It's the smoothest office-style agent add-on I've tested, and it feels like a much more fun version of Paperclip.

How the StarNet AI agent works

The README says the station is a projection of live runtime state, so the picture on screen is a picture of what the agents are really doing.

A room is a team with certain capabilities, a hallway is a handoff lane, and a placed object is a real capability grant.

The agents make real model calls, use real tools and run up real costs on a paid model.

Each agent has its own instructions, memory, history and permissions, and you decide whether it asks for approval or runs with full power.

The project's core rule is that the interface must never show a state the harness cannot prove.

That rule matters, because it means a busy-looking agent is genuinely busy rather than a decorative animation.

How to set up a StarNet AI agent

You either download the desktop app for Windows or macOS, or you run it from source with Node.js 18 or newer.

On my machine the source route still needed an npm ci first, even though the README calls it zero-install.

The full install, including the repo, requirements, licence and updates, is in my StarNet GitHub guide.

Once it opens, creating an agent goes like this.

StepWhat you do
1Give the agent a name and choose its pixel-art appearance.
2Pick a personality from composed, warm, blunt, dry, unhinged or upbeat.
3Choose ask for approval or full power.
4Connect a brain by signing in or pasting an API key.
5Pick the model and a reasoning level of low, medium, high or max.
6Create the agent and watch the waking-up animation.

My agent woke up and said, "I'm awake. Let's give the station a direction."

That took one or two clicks once the brain was connected.

How to use every StarNet feature

The station view lets you zoom in and out, change the frame and go to full zoom.

Everything you run lives in four docks that you can open, show all at once or drag around.

The agent detail panel holds the model, the personality, the instructions, the access permissions for the web and local files, and the approval prompts.

It also has Brief, Growth, History and Memories tabs.

In chat, my agent asked, "What made you want to set up an AI agent?", and I told it SEO and gave it aiprofitboardroom.com.

Chat shows online status and level, the left-hand list switches between idle and online agents, and you can manage projects and sessions.

The plus button adds widgets such as connected apps, pinned widgets and a daily brief.

The app catalog, labelled add an ability, has Gmail, Notion and GitHub, plus finding flights, sending physical letters, email and web search through Firecrawl.

The skill market has originals such as ad copy testing and calendar scheduling, and the community can push updates.

The framework library holds procedures like a decision framework, make a plan, humanising content and hard conversation, and you enable or disable each one.

Crew and Recruit give you roles such as strategist, chief of staff, researcher and marketer.

Deploy overwrites an existing agent with that role, while Summon adds a new agent standing next to yours.

Tasks, quests, a work list, builds and a Systems menu all sit in one system.

The README adds messaging from Telegram, Discord, Slack, Signal and Matrix, a Night Shift mode, cron schedules, an OUTBOX for finished files, MCP connectors and real spend ledgers.

Coming from Hermes Agent?

A viewer was frustrated that StarNet can't use Hermes, and Hermes isn't a brain option.

The README says StarNet can import an existing Hermes or OpenClaw agent, bringing over its persona, instructions, memory and model.

API keys don't transfer, so you re-enter them in StarNet.

Honest limits of the StarNet AI agent

In my video I said I think it's fun to play with, and I don't think there's a massive amount of practical use cases yet.

There's a slight delay when you send a message, so give each reply a moment.

The source install isn't quite as zero-install as the README suggests, because mine needed npm ci.

On a paid model, every agent turn costs real money, so a busy crew can run up a bill.

The README says spend and budgets are tracked on disk and shown as they are, so check them before you leave a crew running.

Local models are free, but the README is honest that they are slower and rougher on long tasks.

StarNet doesn't ship its own model, so pick the brain

StarNet does not come with its own model, so an agent is only as good as the brain you plug into it.

StarNet supports sign-ins for Grok, Kimi, ChatGPT through Codex and the Claude CLI, which uses a Claude Code subscription.

It also takes API keys for OpenRouter, Anthropic, OpenAI, Gemini, xAI, Groq, Mistral, DeepSeek, Together, Fireworks, Perplexity and Cerebras.

For free use it runs local models through Ollama.

When I set mine up, I signed in with Grok and picked Grok 4.6 over Grok 4.7, because I actually prefer 4.6 for this kind of chat.

To help you choose, I'll use GoldieBench, the leaderboard where I score models on real one-shot builds.

Here's how the brains you're most likely to plug in score on the live board.

Brain you can plug into StarNetGoldieBench averageScored tasksHow you connect it in StarNet
Claude Opus 58.2750Claude CLI or an Anthropic key
GPT-5.6 Sol8.1650An OpenAI key, or OpenRouter
Claude Fable 58.1047Claude CLI or an Anthropic key
Grok8.0943Grok sign-in or an xAI key
Kimi K37.8950Kimi sign-in
Claude Opus 5.57.5750Claude CLI or an Anthropic key
Kimi K2.77.4647OpenRouter
Grok 4.77.1520Grok sign-in or an xAI key
Gemini 3.6 Flash7.0850A Gemini key
Claude Sonnet 57.0147Claude CLI or an Anthropic key
Grok 4.65.9520Grok sign-in or an xAI key

One caution before you read that table like a league.

The two Grok 4.x rows come from twenty skill-infused game briefs, while most of the other rows come from the full task set.

So compare Grok 4.6 with Grok 4.7 directly, and treat the rest as a wider guide.

Grok 4.6 vs Grok 4.7 inside a StarNet AI agent

This is the comparison I get asked about most, because I picked the lower-scoring one.

On GoldieBench, Grok 4.7 averages 7.15 and Grok 4.6 averages 5.95 on the same twenty game briefs.

Grok 4.6Grok 4.7
GoldieBench average5.957.15
Scored game builds2020
Medals1 gold2 silver and 3 bronze
Head-to-head on the same briefsIt lost 14 of the 20 match-ups to Grok 4.7.It won 14 of the 20 match-ups.
Price on the standard tier$2 in and $6 out per million tokens$2 in and $6 out per million tokens
Context window500,000 tokens500,000 tokens
Standout strengthIt repaired 4 of 4 failed games on the first retry when handed the real error.Its best one-shots were neoncity at 8.7, racing at 8.7 and arcade at 8.6.
Known weak spotFlight sims shipped unflyable on the first pass.2 of 20 builds died on load or never moved when played.

So why do I still prefer Grok 4.6 inside StarNet?

GoldieBench measures one-shot builds, where a model has to ship a whole game in a single HTML file.

A StarNet agent mostly chats, plans, calls tools and takes small actions with you watching.

That's a different job, and the way a model feels in conversation matters more there than how it does on a 40KB game file.

Grok 4.6's self-repair record also suits an agent loop, because agents hit errors and need to fix them.

If your StarNet crew is going to build things, the bench clearly says pick Grok 4.7.

If it's mostly going to talk with you and run small jobs, test both and keep the one you like, because they cost exactly the same.

Local brains for a free StarNet AI agent

The README suggests installing Ollama and pulling a model such as qwen3:8b for a free, private agent.

That exact model isn't on GoldieBench, so I won't pretend to have a score for it.

Here's what the local board says about local models that are on it.

Local modelGoldieBench averageWhat it means for a StarNet agent
Qwable 5 27B Coder7.14It's the strongest local builder on the board, but it's heavier to run.
Agents-A14.83It's agent-tuned and fast, which suits tool-calling loops.
Gemma 4 12B MLX3.98It's light enough for a small machine but much weaker on builds.

The README also says an 8B model needs about 10 GB of graphics memory while it works, and the model must support tools.

If you want the best results, keep a cloud brain for the agents that do real work and use local models for private, simple chores.

Matching the brain to each crew role

Here's how I'd use the scores when you recruit a crew.

Crew roleWhat it doesBrain I'd test firstWhy
StrategistPlans and makes decisionsClaude Opus 5 (8.27)It's the highest single model on this list.
ResearcherGathers facts with web searchKimi K3 (7.89)It's tuned for long-horizon agent work and has a 1M context.
MarketerDrafts copy and ideasGrok 4.6 or Grok 4.7Both cost $2 in and $6 out per million tokens, which suits high volume.
BuilderShips pages or small appsGrok 4.7 (7.15)It won 14 of 20 head-to-heads against 4.6 on builds.
Private helperHandles sensitive local filesQwable 5 27B (7.14)It's the best local option on the board.

These are starting points, not rules, so swap brains in the agent panel and watch the History tab.

How to test two brains side by side in StarNet

You don't have to trust my table, because StarNet makes it easy to run your own small test.

First, create two agents with the same personality, the same instructions and the same permissions.

Second, connect one to Grok 4.6 and the other to Grok 4.7, or to any two brains from the table above.

Third, set both to ask for approval so neither can act without you seeing it first.

Fourth, give both agents the same small job, such as drafting three content ideas for your website.

Fifth, open each agent's History tab and compare what it actually did, not just what it said.

Sixth, keep the brain that did the job better and swap the other agent onto it in the detail panel.

That's the same idea behind GoldieBench, which gives every model the same prompt and judges the result.

The difference is that your test uses your real job, so it tells you something the board can't.

Two agents running the same job means you pay for both calls on a paid model.

If you're on a subscription sign-in like Grok, the cost question changes, so check your plan's limits before you run a big test.

What GoldieBench does not tell you about StarNet

GoldieBench scores models, not harnesses.

I haven't benchmarked StarNet itself, and a station score would be meaningless anyway.

The board can't tell you how smooth the chat feels, how good the permissions panel is or how fun the levelling is.

It also can't measure the slight chat delay I noticed.

What it can tell you is which brain is most likely to ship working output when an agent builds something.

You can compare any of these models side by side on the compare page, and the scoring rules are on the methodology page.

Pick the brain from the scores, test it on a real job, and keep approvals on while you get to know your StarNet AI agent.

FAQ

What is the StarNet AI agent?

StarNet is a free, open-source, local-first desktop harness where you create real AI agents and watch them work inside a pixel-art space station. It supports sign-ins such as Grok, Kimi, ChatGPT through Codex and the Claude CLI, plus API keys and local Ollama models.

Does StarNet come with its own AI model?

No. StarNet does not ship its own model. You connect a brain by signing in with Grok, Kimi, ChatGPT through Codex or the Claude CLI, by pasting an API key, or by running a free local model through Ollama, and that choice decides how capable your agents are.

Is Grok 4.6 or Grok 4.7 better for a StarNet AI agent?

On GoldieBench, Grok 4.7 averages 7.15 and Grok 4.6 averages 5.95 on the same twenty game briefs, and 4.7 won 14 of 20 head-to-heads. Julian still prefers Grok 4.6 for chatty station agents, and both cost $2 in and $6 out per million tokens, so test both.

Which brain scores best on GoldieBench for StarNet?

Of the brains StarNet can connect, Claude Opus 5 scores highest at 8.27, followed by GPT-5.6 Sol at 8.16, Claude Fable 5 at 8.10 and Grok at 8.09. Kimi K3 scores 7.89.

Is the StarNet local model on GoldieBench?

The README's example, qwen3:8b through Ollama, is not on GoldieBench. The best local model on the board is Qwable 5 27B Coder at 7.14, ahead of Agents-A1 at 4.83 and Gemma 4 12B MLX at 3.98.

Does GoldieBench score StarNet itself?

No. GoldieBench scores models on one-shot builds, not agent harnesses. Use it to choose the brain you plug into StarNet, and judge the harness on your own workflow.

The same stack Julian uses

Run this stack yourself.

Every demo on this bench was built inside the Agent Operating System — one prompt, one shot, single HTML file out. The Agent OS, the prompts, the templates, the weekly walkthroughs and 3,400+ founders shipping with it every day all live inside the AI Profit Boardroom.

3,400+founders
258documented wins
38countries
$49/momonthly