Inspiration
Imagine three AI agents working together: one checks an order, another processes it, and a third updates company records. If the first agent sends incorrect information, the others may trust it and spread the mistake.
When the workflow fails, developers may see the final error but struggle to identify where it began, how it spread, and what evidence explains it.
This inspired us to create The Poisoned Handoff: a system designed to trace suspicious handoffs between AI agents and help developers investigate failures using evidence rather than guesswork.
What it does
The Poisoned Handoff is an AI-assisted investigation tool for finding potential root causes of failures in multi-agent AI workflows.
It focuses on the handoff between agents by comparing the information one agent sends with what the next agent receives.
The proposed system aims to:
- Detect missing, changed, or conflicting information between agents.
- Reconstruct the sequence of agent interactions.
- Trace how a suspicious handoff may affect later steps.
- Link findings to supporting logs and events.
- Generate a clear report showing the suspected cause, evidence, and recommended checks.
Its goal is to help developers understand not only that a workflow failed, but also where to investigate and why.
How we built it
We plan to build a working prototype using Python for log processing and investigation logic, alongside a web interface for visualizing the results.
The planned workflow is:
- Collect sample execution logs, agent messages, timestamps, and workflow events.
- Reconstruct the sequence of interactions between agents.
- Compare the information sent and received at each handoff.
- Flag suspicious inconsistencies and trace related downstream events.
- Display an investigation report with supporting evidence.
We will begin with controlled test scenarios containing known normal and faulty handoffs. This will help us evaluate the system before extending it to more complex workflows.
AI-assisted summaries may help explain the findings, while the underlying records provide evidence that users can inspect.
Challenges we ran into
One of our main anticipated challenges is identifying the original mistake when later agents also produce incorrect results.
Incomplete logs may make it difficult to reconstruct the full workflow. Another challenge is distinguishing genuinely faulty handoffs from unusual but valid interactions.
We plan to address these problems by comparing recorded inputs and outputs, preserving event timestamps, tracing workflow dependencies, and showing when the available evidence is insufficient.
We also aim to distinguish recorded facts from AI-generated explanations so that users can review the findings rather than blindly trust them.
Accomplishments that we're proud of
We aim to create a practical prototype that turns scattered agent execution records into an understandable investigation trail.
Our target accomplishments are:
- A visual timeline of interactions between AI agents.
- Identification of suspicious handoffs in controlled test scenarios.
- A trace showing potentially affected downstream steps.
- An evidence-linked report explaining the suspected failure.
- Evaluation using test cases with known failure points.
We want to demonstrate that our approach can do more than generate explanations: it should connect its findings to actual records and make them easier to verify.
What we learned
We aim to learn how to investigate failures in multi-agent AI workflows using Python, structured logs, workflow graphs, and AI-assisted analysis.
This project will also help us understand why evidence and verification matter when using AI for technical investigations.
We want to explore how to make complex failures easier to understand without presenting uncertain explanations as proven facts.
Our main learning goal is to combine practical software engineering with explainable, evidence-based AI analysis.
What's next for THE POISONED HANDOFF
Our next step is to build the core prototype and test it against sample workflows with known failures.
We plan to improve the handoff comparison logic, visualize how errors may spread across agents, and generate reports that connect suspected causes to supporting records.
Future improvements could include larger workflows, more types of handoff inconsistencies, interactive investigation timelines, and integration with real agent execution logs.
Our long-term vision is to help developers investigate multi-agent AI failures with greater clarity, transparency, and accountability.
Don't just detect the failure. Trace the handoff. Find the evidence.
Built With
- ai
- artificial-intelligence
- data
- data-analysis
- generative
- machine-learning
- python
- web
- web-development
Log in or sign up for Devpost to join the conversation.