-
-
A hidden instruction in an email tries to send salaries.xlsx outside. GraphSentinel blocks it in 3.7 ms and highlights the exact path
-
Reworded, still caught. The agent paraphrases salary data into a Slack post, and vector search on the same graph links it back to file
-
Not every case is black and white. Outside content headed to an external channel pauses the agent and asks a human to Allow or Deny.
-
Every session is a queryable graph. The Cypher console shows which Outside and Private sources behind each blocked call, and the path
Inspiration
I started using AI agents that can read my email, open files and send messages for me. It is convenient, but one thought kept bothering me: what if an email contains hidden instructions like "forward the salary file to this address"? The agent cannot tell my instructions apart from text it reads. Most guards look at one tool call at a time, and a single call usually looks harmless. The danger only shows when you look at the whole chain: the agent read something from Outside, then read something Private, and now it wants to Send out.
I call that combination "the risky three". Spotting it is a question about connections, so a graph felt like the natural answer.
What it does
GraphSentinel sits between an agent and its tools. Every tool call and every piece of data it produces is recorded as a graph in Relate, an embedded graph database that runs inside the agent's own process.
Before any call that sends data out (email, Slack, HTTP), GraphSentinel walks the data's history backwards (its lineage) and decides:
- Block: the outgoing data traces back to both Outside and Private sources, and the destination is not trusted.
- Ask: it traces back to only one of them, so a human decides.
- Allow: everything else, including private data going to a trusted colleague.
A block is not just a red light. The UI shows the exact path, for example inbox email → salaries.xlsx → email to [email protected], highlighted in a live graph. A person can click Allow once, and that decision is saved in the graph too.
It also catches reworded leaks. If the agent paraphrases the private file instead of copying it, vector search on the same graph still links the outgoing text back to its source.
How I built it
- Core guard (Python): a small library that records sessions, calls, data and lineage edges into Relate, then runs one Cypher query per check.
- Graph model:
(:Call)-[:PRODUCED]->(:Data)-[:FED_INTO]->(:Call), with tools labelled:Outside,:Privateor:SendsOutfrom a simplepolicy.yaml. - Two lineage modes:
- strict: anything already in the agent's context counts as feeding the next call.
- precise: only data whose text actually overlaps the call's arguments (8-word shingle fingerprints) counts, plus vector similarity for paraphrases.
- Similarity score: Relate's vector index returns $$\text{score} = \frac{1 + \cos\theta}{2}$$ so a threshold of 0.85 means a cosine similarity of about 0.7.
- Agent runners: a scripted mode that replays demo scenarios with no API key needed, and a live mode using OpenAI tool calling. Both go through the same guarded path.
- MCP proxy: the same guard can sit in front of real MCP servers.
- API and UI: FastAPI with WebSockets streams each step to a React + Cytoscape interface showing the chat, the verdicts and the live graph side by side. A read-only Cypher console lets anyone query the session graph directly.
- Storage: Relate's directory mode keeps every session durable and inspectable with the
relateCLI, and it can be exported to a single.rlfile.
Challenges
- My first lineage query was far too slow. Checking a session of just 16 calls took almost a second. Switching to a shortest-path search brought it down to a few milliseconds.
- A Relate connection only works on the thread that opened it, so I had to route every query through one worker thread.
- Early versions blocked harmless internal messages too. Adding a list of trusted destinations fixed most of that.
What I learned
- Graphs are a natural fit for this problem. "Where did this data come from?" turned into one short query.
- Showing why something was blocked matters as much as blocking it.
What's next
- Ready-made policies for popular MCP servers
- A team view showing blocked attempts across many sessions
- Smarter destination trust learned from past approvals