Every month I used to lose an evening to accounting.

I run two software businesses through one company. One of them has a business partner, and we both pay for things. So every month there are subscriptions, infrastructure bills, API charges, payouts from the payment provider, refunds, invoices in two currencies, and that one transaction that clearly belongs to another month.

None of it is hard. All of it together is exhausting.

Now I type "prepare the monthly reports" in Slack, attach the bank statement, and check what comes back. It takes about 20 minutes of my attention. (That's my own estimate, not a stopwatch.)

The funny part: the thing doing it isn't a finance tool. It's the same agent that reviews our pull requests.

We call it ThreadSherpa. It lives in Slack.

It checks every invoice (right month, our company details on it, amount not zero), matches card payments to invoices, looks for missing documents in email, pulls the payout breakdown from the payment provider, updates the shared ledger with my partner, and prepares two packages: one for the accountant, one for the partner.

Here is the whole run. The highlighted step is the only one that can need me, and it rarely does: about once every two or three months, usually with one yes/no question.

When it does ask, it is the kind of question a new bookkeeper would ask. Is this new vendor shared between the two businesses? Is this a business purchase or a personal one? This wallet was topped up last month, so should the usage count again? It remembers the answer, which is why the questions got rare.

The hard part was never the spreadsheet. It was remembering the rules: which subscription is shared and since which month, what to do when a charge posts on the 2nd but the invoice says the 31st, how not to count a prepaid top-up twice. Those rules now live in a plain written playbook the agent follows, plus a few small scripts. When a new edge case shows up, I add a line. That's it. No magic.

One more thing I like: one of our products records what each paid API call costs. So when the bill jumps, we can ask why we paid, not only what we paid.

Because we already had a tool for everything, and the work was in between them.

Take a simple question: why did our costs jump last week? A chatbot will give you five generic guesses. A person will open six tabs. We wanted something that goes and checks.

Access doesn't make it right. A log can be incomplete, and a model can still confuse "happened at the same time" with "caused it". But it can check, and it shows what it checked. Before one review it put it better than I can: "If I make any claim about the code, it comes with a verbatim quote and file:line… If I haven't read it, I say 'I haven't checked' — not 'probably,' not 'I believe.'"

We didn't plan an "AI operating system". We connected one thing, hit the next bottleneck, connected the next thing.

More than I expected, and not only developers. These are real requests from that small team's Slack. The requests are quoted as typed and the bot's answers are shortened. Account numbers, emails and names are changed.

Scheduling works the same way. "Can you schedule a current conversion rate for trialing->paid report to be posted in this thread every day in 14:05 GMT+2?" was set up in 24 seconds, backed by a live database query.

How it behaves when it is unsure#

What matters as much as the answers is how it behaves when it is unsure or wrong.

It answers in the language it is asked in, too. A teammate wrote to it in Bulgarian, and it replied in Bulgarian and attached the Slack thread to the right task in our issue tracker.

It has to be pleasant#

Otherwise people stop asking.

Yes, and that's when it stopped being a chatbot. All the counts in this section come from the bot's own logs, not from separate monitoring. A pull request triggers a review. A schedule runs a check and posts the result. An alert starts an investigation.

These are the scripted jobs in one small team's deployment (not my own company; a three-person startup I helped). All-time run counts, as reported by the bot's own logs.

The same team sent 1,415 requests in 30 days (5 people, 7 channels) and got 1,190 completed pull-request reviews over a longer period.

One sample I could count#

Counts of runs say nothing about quality, so here is one sample. I asked the bot to go through the last 30 pull requests it reviewed for that team and report what its first review said.

So it asked for changes or blocked on 22 of 30. Were those problems real? I can only partly say:

• On 16 of the 22, the author pushed new commits after the review. That does not prove the commits fixed what the bot found.
• On 3, the author's own words confirm it: two pull requests are titled as review fixes, and one is a follow-up that closes both findings of an earlier review.
• The median first review took about 16.5 minutes from start to finish (8 to 35). That is elapsed time. A review runs a lot of checks and its instructions are very long, so it is not quick.

Two findings from the sample that anyone on the team can check:

• A migration filtered on an expression the existing index did not cover. The bot ran the query plan on production: cost 14,742 as written, 5.95 with the indexed form.
• One pull request was flagged as a blocker. Its new permission checks would have rejected live partner integrations, and the bot named the three partners whose stored permissions would trigger it.

One job, start to finish#

My favourite of those jobs is the boring one. Someone asked the bot to check cache memory every morning and post it in Slack. Its first run failed. Step through what happened next.

And if the repair had not worked? It gets a limited amount of time and tokens. When that runs out, it stops and posts a message in Slack saying the job did not succeed and asking someone to check it. It does not keep trying. So far that has never happened.

No person fixed anything in this story. The only human message is the request. Nobody opened a project for this. AI wrote the script, AI fixed it once, and since then plain code does the work. Changing the check is one more Slack message.

Because paying a model to rediscover the same steps every morning is silly.

If a task is stable, the agent writes a script once and the schedule runs the script. The model comes back only when something breaks. Then it gets the original request, the old code and the error, and tries to repair the script, with a time and token limit so it can't loop forever. That is what happened on the first run of the cache job.

Yes, and mostly without being told to. There are three kinds of memory, and all of them are plain text I can read.

Slack history#

It indexes every message, and not only the text: it reads text in images, Word, PDF and Excel files, and transcribes voice messages in many languages. Then it searches all of that and links every claim to the original message. The part I trust most is that it says when the answer is not there.

Facts#

After a conversation the bot writes down what it learned, on its own. Almost nobody tells it to.

This is what it looks like from the outside. Someone complains once:

And the next answer for that person comes without the code references. Two more real notes:

• From an investigation: "The 1-hour bounce rate alert produces false positives for accounts that send in large batches, because the alert fires after the batch finishes and divides a late bounce tail by a near-zero send count."
• Another preference: "User wants initial draft messages kept under 40 words, with ability to add more content afterward."

Skills#

When a task with several steps works, the bot saves the procedure and reuses it. When it makes a mistake, it saves the lesson. Follow the path of one task.

A note is one paragraph of text with a label in front. This is the raw form of one:

[SKILL: Email list cleanup segment composition] For list cleanup, create a
segment with 3 conditions using OR logic: 1) Last opened email > 12 months ago,
2) Status = Invalid, 3) Bounce type includes 'Mailbox full'…

Two real notes, one from each branch:

What it is not#

None of this retrains the model. They are notes it reads before it starts, like a colleague checking the wiki. And not every skill is self-taught: a teammate uploaded two style guides for writing help articles, and getting the bot to install them properly took several attempts over three days.

This was the least technical decision and probably the most important one.

Most AI tools are one person talking to one assistant. A company doesn't work like that. Support sees the problem, the developer knows the code, and I remember why we built it that way.

The private chats are an illustration. The Slack thread happened: the bot's line is quoted as written, the two human messages are shortened and the account details are changed. One support ticket turned out to be an outage for every customer, found in nine minutes and fixed fifteen minutes after that.

There's a side effect I didn't expect. People who are not comfortable with AI tools just watched a colleague use it in a channel, added a comment, and a week later were asking their own questions. Nobody had to learn a new app.

Less exotic than it sounds. First, ThreadSherpa is not a product you can buy. I built it for our own companies. One Slack bot on the outside. Inside, a router sends each request to a specialist agent with its own instructions, model and allowed tools. A specialist can ask one other specialist for help, once. We capped it so they don't ping-pong. The routing is our own code, not a framework.

Around that: searchable Slack history, saved facts and rules, scripts, schedules, webhooks. It runs on a small shared server. The models run elsewhere: Claude Opus 5.5 for coding work, Gemini Flash Lite for routing because it is fast, and Gemini Flash for general questions.

On access, two different things. For production data: the specific databases the agents can reach are connected with read-only credentials, limited to what they need. They can look, they cannot change, and they cannot see everything. For code: the bot does write code, but in its own environment with its own test database. When it is done, it sends the change for review. Nothing is deployed without a person. And each specialist only gets the tools on its own list, so the support agent cannot do what the coding agent can.

The manager agent#

There is also a manager agent. A few of us can talk to it to change how the bot behaves: add a specialist, connect a tool, change permissions, even change the bot's own code.

This turned out to be our preferred interface. We built an admin web console too, with all the screens you would expect. In the end we almost never open it. We just message the bot. These are real messages.

What about a crafted message that tricks the bot, also known as prompt injection? We tried to break it ourselves first. Then we ran a careful set of prompt-injection tests against it with Fable 5.1 playing the attacker. Neither got it to do something it shouldn't. That is a result for the attacks we tried, not a proof that none exists, which is one more reason for the list below.

Before you copy this#

What I would keep, and what I would add in a bigger company:

1. Read-only, limited credentials for production data. The agent can look, not change.
2. Its own sandbox for code. A separate environment and a test database, and a person reviews every change before it is deployed.
3. A tool list per agent. No agent gets everything.
4. A whitelist with roles for the manager. Few people, named, and checked by the bot on every message.
5. A budget and a timeout per agent. Ours are set in dollars and seconds, so a confused agent stops by itself.
6. A person reads everything that leaves the company.
7. Add an approval step before the manager changes code or permissions. We don't have one. With more people or more sensitive data, I would.
8. Use API keys, not a subscription. More on that below.

Less than I expected, with a few caveats.

• The server: small and shared with other things. The bot needs under roughly 10 GB of memory.
• General model usage through OpenRouter: under $5 a month.
• A Claude subscription of about $200 a month, shared with my other work.

The subscription is the part that matters. The small team above also runs the bot on regular Claude subscriptions, about $200 a month for the five of them. The bot logs what the same usage would cost if we paid per token through the API instead. That comes to between $500 and $2,000 a month, depending on how many long coding runs there were. Those are numbers from our real usage: we asked the manager agent for them.

To be clear about the two numbers: nobody paid the API figure. It is not my estimate either. Every run reports what it would have cost as a paid API call, and that figure comes from Anthropic's own pricing for the run. The bot adds them up. What the team actually paid that month was the flat subscription. That gap is why we use the subscription.

Here is the same thing as plain arithmetic, for that team of five:

So the API would cost between two and a half and ten times as much for the same work.

One thing I want to be straight about. A subscription is meant for a person working, not for automated agents, and it is not how a company should deploy something like this. We are small companies that are not profitable yet, and this is how we could afford to start. I am not recommending it. For a proper deployment, use the API. For a company with profit, I think the API price is clearly worth it. We are not there yet.

The questions I find more useful than cost per token: how often does this repeat, how much context switching does it remove, and how much checking does it still need from me?

Not much, but not nothing. I haven't measured any of this. It is what I see from using it every day.

1. It asks me about new things. A new vendor, a new kind of expense. That happens about once every two or three months, and it is usually one yes/no question where it already proposes the right answer. It remembers what I said, so the same question doesn't come back.
2. Some documents only a human can find.
3. New jobs can fail on their first run. The bot sees the error and repairs the script itself, like in the cache story.
4. A review can miss a bug. It catches most of what I would have caught myself, and now and then something I would have missed. That is good enough for me, as long as a person still looks at the important changes.
5. Every now and then it reads the right data and draws the wrong conclusion. It is right far more often than it is wrong, but when it is wrong it sounds just as sure. So I read the evidence it shows, not only the conclusion.
6. Nothing leaves the company without me reading it first.

So no, three people did not become twenty. What changed is smaller and more real: fewer repeated investigations, less copying of context between people, and decisions that don't wait for someone to collect the numbers.

When we started, I thought the interesting part was letting an AI use tools. It turned out the interesting part was the tools finally sharing context: finance with product usage, support with code, a Slack discussion with next month's decision.

We never made the model smarter. We gave it context, tools and written-down rules.

If you're building something similar, or you think the admin setup above is crazy, tell me. I'm happy to write a follow-up on the architecture.