TracePilot: Evidence-First AI Incident Investigation

• 

  Incident Command Center

• 

  Research Evidence

• 

  Evidence Ledger and Action Plan

TracePilot: From Incident Reports to Evidence-Based Action

Inspiration

When a service starts failing, teams often have to reason from incomplete reports, scattered documentation, and competing explanations. A symptom appearing after a deployment does not necessarily mean the deployment caused it, and a search result is not proof that a hypothesis is correct.

I built TracePilot around a simple principle: incident investigation should make the evidence, uncertainty, and next steps visible rather than hide them behind a confident AI-generated answer.

What it does

TracePilot is an incident-investigation workspace designed for small engineering teams and service operators.

• Incident management: Create investigations, record affected services and severity, and track progress.
• Live evidence research: Search the web through the Tavily API, inspect real results, and select relevant sources for an investigation.
• Traceable evidence ledger: Save evidence against the correct incident, preserve source URLs, and label externally retrieved material as unverified.
• Structured findings: Organise observations, inferences, and unknowns so hypotheses are not confused with established facts.
• Action planning: Maintain prioritised tasks with rationale, verification steps, and trackable status.
• Persistent workspace: Store incidents, evidence, findings, and tasks in Bolt Database so the work remains available after reload.

The live Tavily integration has been tested successfully and returned five real search results in a verification run. Selected sources can be stored in the evidence ledger and are explicitly labelled as retrieved but not independently verified.

How we built it

I built TracePilot using Bolt with a React and TypeScript application, Bolt Database persistence, and server-side edge functions.

The architecture separates the user interface, incident data, evidence retrieval, and AI investigation service. Tavily searches run through a server-side function using a protected API key. The application validates results, displays source information as text, and persists selected evidence without treating retrieved snippets as verified facts.

I also implemented a server-side integration configured for NVIDIA Nemotron 3.5 Lightning through Nebius Token Factory. It is designed to return structured incident summaries, hypotheses, evidence-linked findings, unresolved questions, and prioritised actions, while retaining analysis history and distinguishing AI-generated records from demonstration content.

Important integration status: Tavily research is live and verified. The Nemotron integration is implemented but has not yet completed a successful live inference request because a usable Nebius Token Factory credential is not yet available. I am not presenting demonstration findings as real Nemotron output.

Challenges and lessons learned

One of the main challenges was keeping the system useful without making unsupported claims. Search results can be relevant without being authoritative, and an apparent correlation between an incident and a recent deployment does not establish causation.

This led me to emphasise source provenance, explicit uncertainty, evidence references, server-side secret handling, input validation, and persistent results. I also designed the analysis workflow to preserve the last successful result if a later attempt fails.

A separate infrastructure challenge is obtaining the required Nebius access. The inference integration is implemented, but access and runtime verification remain unresolved. I am treating that as a real blocker rather than substituting another model provider or inventing successful inference results.

What comes next

The next milestone is to verify a live Nemotron inference request through Nebius Token Factory, validate the generated analysis against the incident's actual evidence, and demonstrate how newly retrieved evidence informs the action plan.

Implementation and infrastructure status

TracePilot's incident management, evidence persistence, and Tavily-powered web research are implemented. The deployed Tavily integration has successfully returned real search results, and selected sources can be saved to the evidence ledger with an explicit unverified status.

The application also contains a server-side integration designed to call NVIDIA Nemotron 3.5 Lightning through Nebius Token Factory. However, a successful live Nemotron inference request has not been verified because I could not obtain a usable Nebius API credential through the available billing-registration process in Pakistan.

I am documenting this limitation explicitly rather than representing demonstration findings as live model output. The Nebius/NVIDIA runtime requirement therefore remains unresolved.

My goal is an investigation tool that helps people decide what to check next, understand why a hypothesis is plausible, and verify whether an action worked—not simply generate another AI summary.

Built With

• agents
• ai
• application
• artificial
• bolt
• bolt.new
• database
• factory
• incident
• intelligence
• nebius
• nemotron
• nvidia
• react
• response
• supabase
• tavily
• token
• typescript
• web