Manage Personalized AI Slack Agents Across Your Org

Manage personalized AI Slack agents across your org

We've been quietly running a per-person AI agent experiment at Echobind where each teammate has their own Slack AI assistant. Each assistant has its own bot identity, its own memory and its own learnings (and eventually it's own token usage tracking). The agents live on a private tailnet so we can ssh in as needed, while provisioning and lifecycle are handled from a small internal tool we built called Roster.

This post is the story of how we built it, why we picked the technologies we did, and what the UX taught us along the way. We want to share what we learned since portions of our work is made public and open-sourced (the Railway template and agent container repo). And If you're thinking about embedding agents into your own team's workflow, this demonstrates what we might build for you too.

The stack overview:

• Hermes - the open-source agent runtime from Nous Research. Persistent memory, skills, learning, multiple chat-platform gateways out of the box.
• rome-on-rails - Echobind's Railway template that wraps the upstream Hermes Docker container with a Tailscale sidecar, an opinionated entrypoint, and a documented env-var contract.
• Tailscale - runs inside the container, so each agent is its own tailnet node we can SSH into, with no public surface.
• Roster - Echobind's internal control platform with an org chart of people plus the agents assigned to them. It contains the "Create new agent" flow that drives the whole Railway provisioning sequence and lifecycle control.

Why per-person agents, and what that meant for the design

We didn't want one agent for the whole company. We wanted personalized agents with there own:

• Slack identity. A distinct bot name, avatar, and Slack app, so the assistant feels personal and their context is not cross-pollinated with other bots.
• Memory. Conversations, skills, and learned preferences that don't bleed across teammates. (Hermes's state.db lives on a per-service Railway volume)
• Access boundary. Only the people on a specific Slack allowlist can talk to a given agent and the list can be updated any time.
• Maintainer surface. When something breaks, we need the ability to ssh in and look at logs and inspect files directly.

Those four constraints shaped almost every decision.

The two-layer access model

One Hermes Railway service per Slack agent, behind tailnet was our chosen model. It lets team members chat with their agent without ever knowing what Tailscale is, while an engineer who needs to debug an agent's memory or rotate a skill can quickly ssh in without exposing anything to the public internet. Decoupling those two surfaces was important and Tailscale turned out to be the right tool for the job.

Screenshot: the Roster 'agents' dashboard.

The Railway template for Hermes Slack agents

Before we built Roster, we built rome-on-rails. It's a Railway template that deploys a single Hermes agent and tailnet node. It pulls the upstream Hermes Docker image (we purposely designed around the upstream image so we can benefit from future updates with minimal maintenance), adds a small entrypoint that sets up Tailscale networking, runs Tailscale with ssh, and then hands off to the upstream Hermes entrypoint. It also persists agent memory on the Railway volume so an agent keeps the same tailnet hostname and memory across redeploys.

A big design decision for the Railway template was to stay deliberately single-service since Railway templates fix the service count at publish time. So a 3-service template is wrong wouldn't work for a client who needs 2 or 4 agents, and publishing a per-count template means maintaining multiple near-identical templates with different service counts. So instead, the template provisions one Hermes service, and to add another agent in the same Railway project, we duplicate the service inside Railway (in our case, programmatically using the Railway API). All we have to do is override the per-agent environment variables (Slack tokens, Tailscale hostname, LLM provider key) but leave Railway workspace token and the Tailscale auth key alone since they are shared at the project level. Per-agent variables on the service with shared variables on the project means we have granularity over control of things like LLM token usage.

Postman as the empirical validator

Before integrating the Railway API into our frontend, we took the "trust but verify" approach to the API docs. We built a Postman collection first, ran every query against the live system, watched the Railway dashboard and the logs update live , and only then translated the verified shapes into our backend. This also enforced our understanding of the API interactions as we iterated our Postman environment and queries until they were successful. It also let us create a Postman collection json artifact customized to our app that we could leverage using AI later on.

The Roster UX

With the API verified and the template stable, we built the Roster side. Roster is the place we manage who has agents and which humans they support. The idea is an org chart of people with Slack agents assigned as supporting resources for those people.

Screenshot: the Provision modal: top half.

The "Create new agent" button on the Agents page opens the provision modal where the user enters the required information. The prompt and notes fields don't ship to Railway, they just live on the agent row in our database. The "Project" toggle is where the operator decides whether to spin up a brand-new Railway project for this agent or add the agent into an existing project.

Screenshot: the Provision modal: bottom half.

This is where the per-agent environment overrides live. The LLM provider key, the two Slack tokens, the home channel, the Slack Allowed Users checkbox list. Only checked users can talk to the agent in Slack and nobody else. The Slack user ID list is controlled in a table that is seeded by our team, so the user picks from real humans, not free-text member IDs.

Once Provision agent is clicked, Roster does the following:

1. Creates a new project or uses an existing project's ID.
2. Generates a per-project Railway token via and persists it in our DB with encryption. This is what lifecycle calls (start/stop/restart) authenticate with afterward instead of the workspace token.
3. Fetches the published Hermes template, adds the user's inputs to setup the Railway environment variables, and deploys the template.
4. Persists the new agent to the database with status set to provisioning.
5. Returns immediately so we aren't waiting for Railway to finish building.

Since a Railway build can take several minutes, we close the modal the moment Railway has accepted the deploy, show the new agent card in the dashboard as "provisioning", and only update the card to "running" after the deployment webhook arrives back at Roster.

Screenshot: the post-provision handoff. The one manual step left.

(Pasted%20image%2020260502131142.png)

There's one manual step left, paste the per-project webhook URL into Railway's dashboard. Railway's GraphQL API doesn't (yet) expose programmatic webhook configuration, we discovered this suring Postman development, planned for it, and made sure the modal hands the URL off as a simple one click copy-and-paste option.

Screenshot: the project detail page, where that webhook URL lives forever.

The project detail page is where the operator can come back and grab the webhook URL again, regenerate it (treat as a credential), or watch the green "Webhook live" indicator confirm that Railway is talking to us. The page also lists every agent in the project with an available link to its detail view.

Agent lifecycle control via buttons

Once an agent is provisioned, it's lifecycle is controlled by just three buttons on the agent card: Start, Stop, and Regenerate token (decommission button is coming in a future revision). Under the hood:

Screenshot: agent card for lifecycle managment

• Start calls the deployment restart endpoint against the agent's last deployment Id. The container comes back with the same volume, so same Hermes session history, same skills, same Tailscale machine identity, same MagicDNS hostname. The agent picks up where it left off.
• Stop calls the stop deployment endpoint. While the container is paused, Slack and Tailscale go quiet, but the data volume is preserved.

Every one of those actions is observable in Roster via the webhook. The status of each agent flips through provisioning → ready → running → stopped based on what Railway actually says happened, not what we think should have happened.

How to use the Railway template yourself

Roster itself is internal Echobind tool, but the parts you need to build the agent side are public:

• The template: github.com/echobind/rome-on-rails has the Dockerfile, the entrypoint, and template-notes that describe the environment variables.
• The Readme: contains the one-click Railway template deploy button .
• The docs: the rome-on-rails repo includes tailscale-setup.md, secrets-guide.md and webhook-setup.md so you can deploy your first agent in no time.
• Hermes upstream: github.com/NousResearch/hermes-agent credit to the Hermes runtime; we just package it.

Common gotchas

• If SLACK_ALLOWED_USERS is left blank, the bot looks online but never responds.
• If SLACK_BOT_TOKEN and SLACK_APP_TOKEN are reused accross services, two containers connect to the same Slack app and there is no way to specify which agent receives the message.
• If TS_HOSTNAME is not unique, machines fight for the same hostname in Tailscale; only one wins, the others are unreachable. Always set an explicit hostname.
• Choose Reusable Tailscale auth keys since non-reusable keys spend themselves on the first registration and the agent can't re-register after a redeploy.
• When testing webhooks in local dev, Railway can't reach your laptop. We used ngrok to provide a tunnel for the webhook to reach our local dev and put the public URL in NEXT_PUBLIC_APP_URL.

Future considerations

A few things we're excited about in the next iteration:

• Programmatic Slack-app creation: Today, every new agent needs a Slack app created by hand (bot token, app token, install to workspace). We're going to explore the Slack App Manifest API to provision Slack apps automatically as part of the "Create new agent" flow.
• Editable allowed-users list: The provisioning modal already has the right checkbox UX for picking allowed Slack users. We're going to show the same control on the agent detail page so moving users between existing Slack agents is easy.
• Per-agent token usage and live cost: OpenRouter and the upstream providers expose usage data per request. We want to easily see how much agents have been spending and what their allowed to spend in Roster. For now the spend column is seeded, but we hope to have this live soon.

If any of this is interesting to your team, whether that's "we want our own per-person agents," "we want to build something like Roster for our own runtime," or just "we want to talk through whether AI agents are the right shape for our workflow", we'd love to hear from you.

And if you want to play with the agent side without us, the template is available. Deploy it, paste in a Tailscale auth key and a Slack bot tokens, and you'll have your own personal Hermes assistant on your tailnet in minutes. The hardest part is picking what to name it.

Special thanks to Nous Research for the open-source Hermes runtime that this is all built on, and to the Railway and Tailscale teams for platforms that enable us to experiment with new ideas.