BackWe Built a Slack AI Agent on Hermes. Here's Everything That Broke.
During my first internship in 2012, a coworker shared the company's "acronym bot" to help me get up to speed. It was a large company with a LOT of acronyms, and I still remember what a lifesaver that integrated chat bot was. Fast forward to 2026, and the value of a customized AI-powered chatbot within an organization goes well beyond fetching acronym definitions. Today's AI assistants can perform complex tasks using natural language prompts, schedule and perform recurring tasks, create deliverables, or even change their behavior under the hood for a better user experience.
Today we'll implement a custom AI-powered Slack app that responds when @mentioned inside a channel it's been invited to. We'll use an open source AI framework called Hermes Agent to power our app. This is our framework of choice because it's lightweight, works with our LLM of choice including a local LLM, and because it supports the features mentioned above. Tailscale will serve as our private network provider to ensure our server is only accessible by us.
Spin Up a Linux Server and Lock It Down
Before we can install anything, we need a server that's always on and reachable by Slack. A cloud virtual machine is a great fit for a proof of concept project of this scope. We'll use a DigitalOcean Droplet to host our Linux server with the 2GB RAM option. This gives us some headroom on top of required processes (Linux, Hermes framework, observability dashboard, SQLite session, etc.) which would creep dangerously close to the 1GB plan. The setup is straightforward. We pick a region, an operating system, CPU and storage specs, paste our SSH key, and we're ready to go.
For the OS, we're going with the Ubuntu long term support image since Hermes docs use Linux commands throughout their documentation and the FAQ section even recommends Ubuntu. Once the Droplet is created, we SSH in, create a non-root user with sudo privileges to work from, run sudo apt update && sudo apt upgrade -y to get everything current, and reboot. Now we've got a fresh server on DigitalOcean with a public IP address.
A fresh Linux server accepts connections on any port by default. We want to make sure SSH key authentication is the only way in. DigitalOcean handles this when we create a Droplet using an SSH key by disabling password login automatically. We'll use Tailscale to limit who can attempt SSH key authentication. Tailscale creates a private encrypted network between our devices using the WireGuard protocol. Think of it like a VPN, but we don't run a VPN server. Instead, Tailscale handles the coordination, and the actual traffic is encrypted end-to-end between our devices. We install it on the server, install it on our laptop, sign in with the same account, and now they can talk to each other over private IPs that nobody else can reach.
Now with Tailscale up, we configure the server's firewall (ufw on Ubuntu) to deny incoming traffic by default. Then create an exception to only allow traffic arriving through the Tailscale interface. Outbound traffic is allowed since the server needs to call external APIs (Slack, our LLM provider, package repositories). So now even if someone knows our server's public IP address and tries to connect on any port, the connection gets dropped.
sudo ufw allow in on tailscale0 sudo ufw allow 41641/udp sudo ufw default deny incoming sudo ufw default allow outgoing sudo ufw enable
Set Up the Hermes Agent Framework on the Server
Hermes Agent is an open source AI agent framework that gives our LLM tools persistent memory and messaging integrations. It extends chatbot capabilities to executing tasks, scheduling cron jobs, searching the web, reading files, and remembering context across conversations. It supports multiple LLM providers including local models. For our LLM provider, we went with OpenRouter with Claude Sonnet 4.5 as our model. OpenRouter lets us access models from multiple providers (Anthropic, OpenAI, etc.) through a single API.
Installation is a one-liner:
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash
The installer handles everything: it installs Python, clones the Hermes repo, installs all dependencies, and launches a setup wizard. The wizard walks us through picking our provider, model, terminal backend, session settings, and messaging integrations. It was honestly very easy to set up following the wizard, but hand-editing the YAML config is also a valid option. After the wizard finishes, we verify the install with hermes doctor (which checks API keys, tools, and config) and a quick test chat to make sure the model responds.
Troubleshooting: the setup wizard wrote the model config as a flat string.
model: anthropic/claude-sonnet-4.5
But Hermes actually expects a nested YAML key.
model: default: anthropic/claude-sonnet-4.5
When left flat, Hermes falls back to a different default model. In our case it defaulted to Opus 4.6.
Create the Slack App and Connect to the Server via WebSocket
This step follows the official Hermes Slack setup guide. We begin by creating our Slack app in the Slack API dashboard, then configure the bot token scopes, and grab the two required tokens for Hermes. Socket Mode needs to be enabled since we prevent incoming traffic. Without it, Slack is unable to send messages to our server. WebSocket mode allows the server to initiate two way communication with Slack.
With our Slack app set up, we can add the Slack API tokens to our server environment variables and install the Hermes Gateway. We'll install it as a systemd service (hermes gateway install), and enable "lingering" so the service keeps running even after we close our SSH session. The gateway is what actually maintains the WebSocket connection to Slack and routes messages to and from the Hermes agent.
Once the gateway is started, we're ready to test the @mention flow in Slack. We just need to install our app in Slack, and then invite Hermes into a channel. We can designate a Slack channel as Hermes' "home" channel where it will log activity by default.
Troubleshooting: During initial gateway testing, the Slack bot was triggering an "active session" interrupt on every first message after startup and preventing the gateway from responding. After some troubleshooting and verifying the Slack app and Hermes environment were set up correctly, we checked the Hermes Agent GitHub repo. We noticed slack.py had been modified in a recent batch of commits with changes that seemed relevant, so we ran hermes update, which pulled 98 new commits. Then we restarted the gateway, and the issue was gone. This was a great reminder to check for updates frequently when working with source code that is moving fast.
Explore Slack Security Guardrail Options
Hermes comes with a lot of built-in toolsets: web search, cron jobs (scheduled tasks), terminal command execution, file operations, code execution, memory, and more. Out of the box, it's powerful. But in some applications we may not want our Slack bot to run shell commands on the server, manipulate files, or spawn sub-agents. Hermes uses a config.yaml that acts as an allow list, so only the toolsets we choose are available on the platform.
Troubleshooting: Hermes has a global toolsets setting that defaults to [all]. If you don't change this to [], it overrides the per-platform restrictions and makes everything available everywhere. We discovered this when Hermes made code changes via Slack requests even though the Slack toolsets were not listed.
Another powerful layer of security is the SLACK_ALLOWED_USERS environment variable. It is a user whitelist that only allows listed users to make Slack requests to Hermes. Slack user IDs are available under Slack user profiles. Every user ID has to be added to the Hermes environment variables before that person can make requests, including our own.
Hermes also has some baked in security guardrails to protect against other forms of threats. In our use case, the user whitelist along with the toolset config offers enough protection. But it's good to understand the defense in depth offered by Hermes for other applications where guardrails need to be tighter.
Set Up Hermes Agent Context Files
Since we want Hermes to always "know" about Echobind, we need to decide how to set up its knowledge internally by referring to the Context Files documentation. Since the "How Context Files Are Loaded" section mentions the overall context is "injected" into our prompt, we want to minimize the overhead of company knowledge that gets sent with every prompt. To do this, we'll take advantage of the Hierarchical Discovery feature when loading context files.
On startup, it walks recursively from its working directory and concatenates every AGENTS.md file it finds to build its context. So we'll use a two-layer approach. A condensed summary of Echobind will live in ~/.hermes/knowledge/AGENTS.md. This gets automatically loaded into every prompt and gives Hermes an overview of the company along with where to find more detailed information. The full company content, scraped directly from Echobind.com in this case, lives as individual markdown files in the same ~/.hermes/knowledge/ directory. When a user asks a specific question, like "tell me about the Stripe partnership", Hermes uses its read_file tool to pull the relevant files that don't get loaded with every prompt.
Troubleshooting: Hermes also maintains its own ~/.hermes/AGENTS.md file where it writes self-improving context over time in the root directory. This file doesn't exist at initial setup, and so the hierarchical discovery fails until it is either manually created, or Hermes creates it. Also, we don't want our Echobind knowledge to be overwritten, so we don't store it in the root directory AGENTS.md file since Hermes updates this as its context updates. We instead store it in the knowledge directory (created by us) AGENTS.md file where it is discovered hierarchically.
Finally we need to populate Hermes' knowledge with content from Echobind.com. We'll use a custom Python scraper to pull web content around services, case studies, capabilities, and partnerships. These are the pages that answer the question "what does this company do?" The final knowledge directory created by the scraper looks like this (each section contains relevant markdown knowledge files).
Test Persistent Knowledge and Memory
This is the moment of truth. Does Hermes actually use its knowledge instead of searching the web? We'll test from the command line first with a request for general information.
hermes chat -q "What does Echobind do?"
Hermes should pull from the scraped content without using tool calls at this point since the Echobind summary in AGENTS.md will handle this.
Next, let's test something more specific, like "Give me a detailed description of Echobind services." This information lives in an individual knowledge file, not in the summary AGENTS.md. So Hermes will use its read_file tool to pull the relevant file(s) and answer with details about the services. This will confirm the two-layer system is working since the summary provides the general overview, and the detailed files get pulled on demand when specifics are needed without searching the web. Let's test this using Slack to get the full user experience.
Over time, Hermes also builds up its own learned context in its self-managed AGENTS.md. As it handles more questions, it adapts by recognizing patterns across sessions and updating how it handles similar questions. This is part of the "adapts to you" piece. The knowledge gives it context, but the persistent Hermes Memory lets it get better at using it. Users can also explicitly tell Hermes to adapt with prompts like, "always format your responses to me using bullet points" or "use relaxed but professional language in this channel".
The last toolset we'll look at is schedule_cronjob, one of the most useful tools Hermes offers. We'll have it perform a simple task here for the purpose of this walkthrough. But it is capable of scheduling and executing complex recurring tasks. Let's ask Hermes to remind us to check our email in 5 minutes.
Create an Observability Portal
We need to know if the agent is working, what it's been doing, and what went wrong when something breaks. Without a dashboard, the only way to check on Hermes is to SSH into the server and review logs manually.
Instead, we'll build a lightweight Flask dashboard since we're already using Python and Flask is a Python web framework. The dashboard is simple with only two routes. The root (/) serves the HTML page, and /api/status returns a single JSON payload with everything the frontend needs. The frontend fetches /api/status on a timer and refreshes the page every 30 seconds. No database, no WebSocket, no client-side framework, just a page that polls a JSON endpoint.
The dashboard will show whether Hermes is up or down, system resource usage (memory, disk, uptime), recent activity parsed from the gateway logs, recent errors and warnings, and the current state of the knowledge files. We'll also set it up to run as its own systemd service that automatically starts up, and make it only accessible over Tailscale.
Fun Fact: We built the initial observability dashboard, then enabled file and terminal command toolsets for Slack to experiment with them. When prompted, Hermes updated the dashboard to add additional observability without any issues. We probably could have let it build the initial dashboard as well.
Final Thoughts
Hermes is a long way from the acronym bot I used during my internship over a decade ago. But the core idea is the same. It's an integrated tool that lives where your team already communicates and helps them get things done. The difference now is that the tool can reason, remember, adapt, and perform tasks instead of just looking up definitions.
We went from an empty DigitalOcean Droplet to a fully functional AI Slack assistant that is secure, connected to Slack, and knows about our company. We also highlighted some troubleshooting and design considerations along the way. Hermes Agent is currently a quickly moving project, but the barrier to entry is not as high as it first seems.
What it costs
- Hermes Agent is open source and free.
- DigitalOcean Droplets start at $12/month for the 2GB tier we used.
- Tailscale has a free tier that's more than enough for a setup like this.
- LLM usage is the variable line item. Token spend scales with how often your team @mentions the bot, so watch it for the first few weeks.
Resources
