BlackBox AI

Beyond the error: discover the why.

It's 2:14 a.m. and a customer's payment has just failed.

The alert says 500 Internal Server Error. That's all it says. Somewhere in the three services that touched that payment, a clue is sitting in a log file, but it's one line among a few hundred thousand, written in a format nobody agreed on, with a timestamp that's off by a few seconds.

So you do what every developer does. You open six tabs. You grep. You squint. You build the timeline in your head, and about forty minutes later you find out the real problem started with a deploy an hour before anyone noticed.

The error was never the story. It was the last line of it.

Why "BlackBox"

When a plane goes down, nobody asks the flight recorder for its best guess. Investigators read it. They rebuild the minutes before the failure from what was actually recorded, and when the recording has a gap, they say so.

We wanted that for software. A tool that doesn't just say "this looks like a database issue" in a confident voice, but one that opens the evidence, lays it out in order, and shows its work.

What it does

You give BlackBox AI the logs from your services. It gives you back a case file.

The timeline. Scattered logs, merged and put in order, so the incident reads like a sequence instead of a pile.

The map. Events and services become a graph, so you can see how one small failure spread outward, and where it began.

The suspects. An AI model proposes root causes, ranked by how well the evidence supports them. Every one of them points to the exact log lines it relies on. If it can't point, it has to call the idea a guess.

The alibis. It also tells you which explanations it considered and dropped, and why. A good investigator is as interested in what didn't happen.

The gaps. Where the evidence runs out, it says so, and tells you what to go check next.

The whole thing lives in an interactive dashboard, so you can follow the reasoning yourself instead of taking its word for it.

A sample case

From one of our simulated incidents.

Symptom: checkout payments fail for twelve minutes.

Confirmed by the logs: the payment service starts timing out at 14:02. The database connection pool hits its limit at 14:01. A deploy to the orders API went out at 13:58.

Hypothesis, not proven: the deploy introduced a connection leak. The timing fits. Nothing in the logs shows the leak directly.

Considered and set aside: a network fault between services. No dropped packets, no DNS errors anywhere in the window.

What's missing: database metrics from before 14:00. Check next: connection counts before and after the deploy.

Notice the shape of that. Three things we know, one thing we suspect, one thing we ruled out, and one thing we'd need to see. That's the format we want every investigation to have.

Under the hood

logs from many services
        |
   parse + normalize          (Python)
        |
   one ordered timeline
        |
   event / service graph      (graph analysis)
        |
   language model reasons over structured evidence
        |
   hypotheses, each tied to cited log lines
        |
   dashboard: timeline, graph, evidence panel

We're starting with simulated incidents: payment failures, API errors, application crashes. We chose them on purpose. When you've planted the cause yourself, you can finally ask whether the tool found it.

The hard part

Logs lie by omission. They're incomplete, noisy, occasionally contradictory, and wildly different failures can look the same from the outside.

Language models add their own problem. They're excellent at sounding sure. A wrong explanation delivered fluently is worse than no explanation, because people believe it.

So we built our rules around that:

No claim without a citation to a real log record.
Competing explanations stay on screen. Nothing gets quietly dropped.
If the evidence isn't there, the answer is "we can't tell yet," followed by what would settle it.
We don't grade ourselves on how convincing the output feels. We grade it against incidents where we already know the answer.

How we'll know it works

We'll run the same set of simulated incidents through two setups: a language model reading raw logs, and BlackBox AI with its graph and structured evidence. Then we'll count: