Inspiration
Have you ever seen an AI agent fail at the final step and wondered where things actually went wrong?
When an AI agent performs a sequence of actions, the final failure doesn't always tell the whole story. Sometimes, the real mistake happens much earlier. An agent might misunderstand a user's request, make a decision based on that misunderstanding, and then execute everything as planned, but still produce the wrong result.
I wanted to explore this problem and build something that helps developers look beyond the final error. That's how CausalTrace Lite came to life: a visual debugging tool that traces an agent's failure back to its earliest incorrect decision.
What it does
CausalTrace Lite helps developers understand why a multi-step AI agent fails instead of simply pointing to the last incorrect action.
My current prototype focuses on a restaurant-booking agent with three stages: Understand, Choose, and Execute.
It demonstrates three failure scenarios:
- The Misunderstood Request: The agent interprets "tonight" as "tomorrow," causing the mistake to propagate through later stages.
- The Wrong Restaurant: The agent understands the request correctly but chooses a restaurant with the wrong cuisine.
- The Booking That Failed: The agent gets everything right until the execution stage submits the wrong party size.
Users can switch between scenarios, inspect individual stages, and compare expected behavior with actual behavior. The dashboard also compares CausalTrace Lite's diagnosis with a naive debugger that focuses on the final erroneous action.
My goal is simple: help developers identify where a failure began, understand its consequences, and debug multi-step agent workflows more effectively.
How I built it
I built CausalTrace Lite as a lightweight web application using HTML, CSS, and JavaScript ES modules. I chose a retro, early-2000s internet-inspired visual theme to make the interface distinctive while keeping the technical information readable.
The core of the application is a structured evaluator that compares expected and actual values at each stage. Rather than assigning a root-cause label directly to each scenario, the evaluator derives the earliest incorrect stage from those comparisons and uses explicit causal relationships to classify downstream effects.
I also implemented a separate naive diagnosis to demonstrate the difference between identifying the final erroneous action and locating the earliest incorrect decision.
The application uses deterministic local data, so it runs without API keys, external services, or third-party packages. I used Node.js's built-in test runner to verify the diagnosis logic across the three scenarios and additional edge cases.
Challenges I ran into
One of my biggest challenges was distinguishing the stage where an error begins from the stages affected by that error.
A downstream stage might behave consistently with the information it received, even when that information was already wrong. Simply marking every later stage as incorrect would make the diagnosis misleading. I needed to compare structured values and represent causal relationships explicitly.
Another challenge was making the comparison meaningful. I wanted to demonstrate a genuine difference between root-cause diagnosis and a naive last-error approach, rather than just displaying two predetermined labels.
I also had to balance functionality with design. The dashboard needed to be visually engaging without making the execution trace, mismatched values, and diagnosis results difficult to understand.
Accomplishments I'm proud of
I'm proud to have turned an abstract debugging problem into an interactive, runnable prototype.
CausalTrace Lite includes three distinct failure scenarios, a structured root-cause evaluator, downstream-effect classification, an independent naive diagnosis, and an interactive dashboard for inspecting agent stages.
I also created automated tests for the diagnosis behavior, including root-cause identification, affected-stage classification, mismatched fields, explanation-text independence, and a fully correct trace.
Another accomplishment is that the prototype runs locally without API keys or external dependencies. This allowed me to focus on the core debugging concept and make the demonstration reproducible.
Finally, I gave the project its own visual identity with a nostalgic Y2K-inspired interface while keeping the debugging information central to the experience.
What I learned
This project helped me think more carefully about how errors propagate through multi-step systems.
A failed final action does not necessarily mean the final stage introduced the problem. To diagnose an agent effectively, I need to understand what each stage received, what it was expected to do, and how its output influenced subsequent decisions.
I also learned the importance of separating evaluation logic from the user interface. Keeping the diagnosis engine independent made it possible to test its behavior without relying on how the dashboard presents the results.
Finally, I learned that a useful prototype doesn't need a complicated tech stack. A focused problem, explicit data structures, deterministic scenarios, and meaningful tests can demonstrate an important engineering idea without unnecessary infrastructure.
What's next for CausalTrace Lite
CausalTrace Lite is currently a deterministic proof of concept, and I see several ways to extend it.
Next, I'd like to explore tracing real agent executions, integrating with tool-calling workflows, and capturing structured inputs and outputs at each stage. I also want to investigate more complex failures involving branching decisions, retries, tool errors, and multiple interacting causes.
Another direction is to improve the visualization of causal dependencies so developers can follow how an early mistake affects later actions across longer workflows.
My long-term goal is to explore how CausalTrace Lite could become a practical debugging aid for developers building multi-step AI agents, helping them understand not just what failed, but where and why the failure began.
Log in or sign up for Devpost to join the conversation.