Mekael Wesley-Rosa

Building an AI coworker that knows what happened

How roughly 380 real questions became 31 reviewed data lookups—and why the agent still cannot ship a change by itself.

Mekael Wesley-Rosa Runwise

This one waits for the guestbook. Leave your name and email and it opens.

Still shut after that? Email me and I’ll send it another way.

A Slack thread on an iPhone. A coworker asks about unusually low flow at three-riser buildings, and Alakazam replies that it is checking flow and connectivity data.
A reenactment of the Slack experience. Alakazam reports what it is checking, then answers in the same thread. Open the replay full screen.

Someone at Runwise asks why the heat did not turn on last Tuesday at six. The answer exists, but it is split between the code that decides when heat should run and the building data that records what actually happened.

Finding both used to mean finding an engineer, asking them to stop what they were doing, and waiting while they translated the system into an answer. A technician in a boiler room and a salesperson on a customer call had the same problem: the company knew the answer, but the person who needed it could not reach it.

So I built Alakazam, an AI assistant in Slack. It started as a way to ask the product a question. It became a way to compare what the product should do with what a building actually did. And eventually, to prepare a fix without being able to publish one.

If I had to pick the single highest-leverage thing I did here, it was not the model or the Slack interface. It was using roughly 380 real questions to define a bounded data layer. More on that in a minute.

5product repositories it reads
31reviewed data lookups
~380real questions behind them
0changes it can publish alone

The questions were reaching engineers

The first problem was not missing information. It was access to information that already existed.

Product behavior lived in five repositories. Production history lived in a data warehouse. The people closest to a customer or a building rarely worked in either place, so engineers became the interface between the company and its own product.

This kind of interruption looks small because each question is small. The engineer usually knows where to look. But the person asking does not know which engineer to find, and a question with no clear owner can wait longer than the investigation itself.

Code was enough, until it wasn’t

The first version of Alakazam read our product code and explained what it found in plain English. That was enough for questions like “How does this setting work?” or “When should this alert appear?”

Then someone asked about a particular building on a particular day. The code could explain the rule, but not whether the boiler was connected, what the sensors reported, or which schedule was active. It knew what should happen. It had no idea what did happen.

One question needs product rules and building data A question enters from Slack. Alakazam checks product rules and building data, then combines them into one plain-English answer. “Why didn’t the heat turn on?” A Alakazam Product rules what should happen Building data what did happen One answer that explains the rule and what happened in this building
Code explains the rule. Building data explains the incident. Most useful questions need both.

It needed another sense, not another brain.

The decision that changed the project

One agent, another source

My first instinct was to build a second agent just for data. An engineer talked me out of it.

Alakazam already understood the product and how its parts fit together. Splitting production questions into another assistant would make the person asking decide whether their question belonged to code or data—the distinction the product was supposed to handle for them.

So I gave the existing agent another source. It could now use product code to explain the rule and building data to reconstruct the event, in the same answer and the same Slack thread.

Real questions defined the data layer

The tempting version of this project gives a model broad database access and asks it to write whatever query seems useful. I did not want usefulness to depend on a plausible query being safe.

Instead, I read roughly 380 questions from real Slack conversations, grouped them by the job each person was trying to do, and turned the recurring needs into 31 reviewed data lookups. Alakazam can choose a lookup. It cannot invent database access.

  1. 1Read the questionsabout heat, equipment, alerts, leaks, and schedules
  2. 2Find the recurring jobsnot every variation in how people asked
  3. 3Build reviewed lookupswith known inputs and bounded results
  4. 4Let Alakazam choosewithout letting it write a query

The tool grew from observed questions, not imagined AI features.

The agent found problems in what it read

Building those lookups exposed something I had not set out to find. Alakazam started catching mistakes in our own assumptions about the data.

One check said there were no critical alerts when there were. Date searches looked at the wrong hours. Equipment health rankings ran in the wrong direction. Each layer above those assumptions—the product, the reviewed query, and the answer—could work as designed and still produce the wrong conclusion.

The fourth layer contained three defects nobody had looked for. Building a reader across the system made them visible.

That changed how I thought about the project. It was not only making answers easier to retrieve. It was forcing old assumptions to answer new questions, and showing us where they could not.

Then people asked, “Can you fix it?”

Once Alakazam could explain a problem, the next request arrived naturally: can you fix it?

I added one more capability. When a person explicitly asks for a change, Alakazam can prepare one in the frontend repository. The edit has to pass the repository’s install, lint, typecheck, and test gates. Then it opens a pull request and stops.

A person still reads and merges the change. That is not a temporary restriction while the agent gets better. It is the product boundary that makes the capability safe to offer.

Each rung widens what Alakazam can do. The fourth is deliberately empty: no prompt or model can publish to production.

The limit is part of the product

Agent demos usually make autonomy look like a finish line. This project taught me to treat it as a budget.

Alakazam reads five repositories because broad context improves an answer. It writes to one because a requested frontend change is a narrow, reviewable task. It publishes to none because the value of skipping that review is small and the cost of a bad change is not.

The useful question was never “How much can the agent do?” It was “What is the least authority it needs to finish this person’s job?” That framing made it possible to expand the product without quietly expanding its power.

The bigger picture

What I want you to take from this is not that we built a clever agent. The most important decision was using real questions to define a bounded data layer. The 31 lookups are useful because the roughly 380 questions came first.

Agent demos usually make autonomy look like a finish line. This project taught me to treat it as a budget. Alakazam reads five repositories because broad context improves an answer. It writes to one because a requested frontend change is a narrow, reviewable task. It publishes to none because the value of skipping that review is small and the cost of a bad change is not.

And to be clear, that last limit is not a temporary restriction while the agent gets better. It is the product boundary that makes the capability safe to offer.

The open question is adoption. Alakazam can do more than most people at Runwise know to ask of it. The next work is not another capability. It is better examples, clearer discovery, and enough hands-on use for the right questions to become obvious.

Role: Product design and implementation · Collaborator: Staff engineer · Company: Runwise