Why I Replaced OpenClaw With a Self-Hosted Hermes Agent and My Own Honcho

I run an always-on AI agent in my private and public cloud infrastructure. It lives on Telegram or WhatsApp, it remembers who I am between conversations, and it has a sysadmin's hands — terminal, code execution, the works. For a while, the engine behind that was OpenClaw. It isn't anymore.

This is the story of why I tore it down and rebuilt on Hermes Agent from Nous Research, backed by a self-hosted Honcho memory layer. Two things forced the decision: a billing change that exposed how fragile my setup actually was, and a security posture I was never fully comfortable with. The rebuild fixed both.

First, let's be fair to OpenClaw

OpenClaw is a genuinely impressive project. It went from nothing to one of the fastest-growing repositories in GitHub history — hundreds of thousands of stars and millions of users in a matter of months. It supports dozens of integrations, routes across multiple model providers, and gets a Telegram-driven agent running in minutes. If you want to talk to an autonomous agent from your phone, it's one of the easiest on-ramps that exists.

So this isn't a hit piece. OpenClaw did exactly what it promised. My problem was never that it didn't work — it was that how it works didn't fit a security-first, vendor-independent operator running infrastructure they actually care about.

What a Self-Hosted Hermes + Honcho Agent Can Actually Do

I run an always-on AI agent in my infrastructure and DevSecOps platforms. It lives on Telegram, it remembers who I am between conversations, and it has a sysadmin's hands. The engine used to be OpenClaw; a billing change and an exposure model I never loved pushed me to rebuild. But this post isn't about the move — it's about what I ended up with, because the combination of Hermes Agent and a self-hosted Honcho memory layer turned out to be far more capable than the sum of its parts.

Two pieces. Hermes is the body — the Telegram gateway, the tools (terminal, code execution, web), the skills, the persona, and a brain that's just a swappable LLM behind an OpenAI-compatible endpoint. Honcho is the memory — but not a fact store. It's a reasoning layer that builds and maintains a model of you. The whole thing sits inside a contained host: nothing connects in, egress is locked to DNS and HTTPS, and the only thing that ever leaves the box is an inference call. Everything else — including the model of me — stays home.

So let's talk about what you can do with it.

It knows you — and gets sharper over time

This is Honcho's headline, and it's a genuinely different experience from "memory" as most agents do it. Most agent memory saves what you said and retrieves it by keyword or vector similarity. Honcho runs a background reasoning pipeline that reads your conversations and derives things: it extracts premises, draws conclusions, and maintains a running representation of how you think, what you care about, and how technical you are.

The agent doesn't just recall facts — it can back-channel with Honcho in plain language and ask, in effect, "what does this person actually want here?" and get a reasoned answer. It uses a peer model where you and the agent are each modeled separately, so the representation reflects the full conversation, not just your half of it.

In practice that means I can start a fresh session weeks later and the agent already knows I want terse answers, that I'm security-first, that I run a Proxmox private cloud— without me re-establishing any of it. The longer it runs, the better its model gets. It's the difference between an assistant that has notes about you and one that has a sense of you.

It's a sysadmin, architect, and PM you reach from your phone

Hermes gives the agent real tools, and a persona file that sets its boundaries. Mine is part sysadmin, part IT architect, part project manager, with a hard rule that it reads and diagnoses freely but never changes anything without my approval — with one narrow emergency carve-out for an active incident.

The first time I asked it to assess its own host, it ran the right commands, produced a clean posture report — open services, missing hardening, pending patches — ranked the findings, and then stopped and asked for explicit sign-off before touching anything, flagging on its own that one fix carried a lockout risk. That's supervised autonomy working exactly as designed: it does the investigative legwork, I make the call. From my phone, anywhere.

Out of the box you get an operator that can:

• triage "why is X slow / unreachable / broken" and report back in plain language,
• audit a host's security posture and prioritize what matters,
• draft and plan work, breaking a problem into steps and tracking it,
• and reach you — not just answer when asked.

It works while you don't

Because it's a persistent service with a scheduler, the agent can act on a cadence, not just on demand. A daily health-and-security summary pushed to my Telegram at a fixed hour. Watchers for disk pressure or a service that didn't come back. A "what changed since yesterday" digest. The proactive direction — the agent messaging me — is half the value of an always-on operator, and it's the half a chat window can never give you.

The brain is a replaceable part

This is the structural win, and the reason the whole thing is durable. The model is configured as a single swappable endpoint. Today the heavy background reasoning runs on a cost-effective model (I use a DeepSeek tier); the configuration is written so I can point it at the Claude API — or anything else — in a few lines, without the agent losing its identity, its skills, or its memory.

That decoupling does two things. It ends vendor lock-in: a pricing change I have no say in can't strip-mine my setup, because the engine is swappable. And it lets me route work by economics — cheap model for routine reasoning, a stronger model for the hard problems — while Honcho carries the continuity across all of it. The agent is the constant; the LLM is a part.

You can teach it. Skills

Skills are codified runbooks the agent follows — your procedures, written down once and executed consistently. Instead of re-explaining how you do a posture check or a release step every time, you author it as a skill and the agent runs it the same way each time. It's how an off-the-shelf agent becomes your operator: the soul sets who it is, the skills set how it does your specific work.

It's the seed of a multi-agent system

Honcho's peer model and Hermes's delegation tools mean this isn't a dead end at one bot. Because memory is peer-based and self-hosted, multiple agents can share one on-prem model of the lab and of me. The natural division of labor: a planner/conductor agent that delegates real coding to a dev tool, executor agents for routine work, all reading the same memory and all reaching infrastructure through a single gated gateway rather than each holding raw credentials.

That last part is the growth path in the diagram — the dashed line. The agent's reach into Proxmox, OPNsense, Cloudflare, Azure, ENTRA ID and GitHub comes through an MCP gateway that holds the credentials and gates the power, read-only first and write access staged carefully behind approval. The agent carries an identity and the right to ask; it never holds the secret itself. You start with an agent that can look at everything and change nothing, and you expand that envelope deliberately.

It's entirely yours

Every piece runs on my hardware. The agent, the tools, the scheduler, and — most importantly — the memory. Honcho's database is the single most sensitive thing in the stack: it's an accumulated model of its operator. That belongs on an encrypted dataset, reachable only over loopback, never on the network. Self-hosting means the agent's understanding of me is mine alone; only the inference call ever egresses, and even that engine is swappable.

The takeaway

Hermes gives you a body with real tools and a swappable brain. Honcho gives it a mind that models you and improves with use. Self-hosting makes the whole thing yours. What you get isn't a chatbot with notes — it's a memory-rich, model-agnostic, self-owned operator that you reach from your phone, that works while you sleep, and that you grow into giving more responsibility as it earns it.

That's a thing worth running for the long haul.

If you're running a similar setup or have questions, reach out in the comments or on LinkedIn.

— Antonio | AboutCloud

arusso@aboutcloud.io