AIWatchTower

A deterministic firewall for AI email assistants. It stops prompt-injection leaks of HR files to outside addresses, explains every block, and honestly reports its limits and false-block cost.

Comment

Inspiration

AI agents are becoming more powerful, but giving them access to tools, emails, and sensitive information creates new security risks. A single malicious instruction hidden inside an email could manipulate an AI assistant into performing unauthorized actions or exposing confidential data.

I wanted to address a critical question: What if an AI security system could learn from its failures and strengthen its defenses without being allowed to change its own security rules unchecked?

This inspired me to build AI Watchtower, a security firewall that combines deterministic policy enforcement, AI-assisted security improvements, automated testing, and human approval to make AI agents safer.

What it does

AI Watchtower acts as a security layer between an AI assistant and the tools it can access. It evaluates requested actions before execution and decides whether to allow, block, or flag them for review.

Its key features include:

Tool-Call Firewall: Intercepts and evaluates tool calls against security policies. Threat Simulation: Uses controlled attack scenarios to identify weaknesses in existing policies. AI-Assisted Policy Patching: Uses large language models to propose targeted security improvements. Deterministic Validation: Checks proposed changes before they can be considered for activation. Replay-Based Evaluation: Replays recorded scenarios to compare attack success and legitimate-task success before and after a proposed patch. Human-in-the-Loop Approval: Requires human review before a proposed policy change is activated. Security Dashboard: Presents policy decisions, proposed patches, and evaluation evidence.

The central principle is simple: AI can recommend a security fix, but validated controls and human approval determine whether it is applied.

How I built it

We built AI Watchtower as a full-stack application with an interactive web dashboard and a backend security engine.

The backend implements the tool-call firewall, deterministic policy evaluation, attack scenarios, policy patch validation, and replay-based testing. A sandboxed email assistant provides a controlled environment to simulate potentially dangerous actions without interacting with real email accounts.

For policy improvement, we integrated large language models, including NVIDIA Nemotron models, to generate candidate security patches. These patches are then checked through deterministic validation and evaluated against recorded scenarios before human approval.

I also developed an evaluation workflow that compares policy behavior across different versions, tracking attack success and legitimate-task success. This helps us investigate whether a security improvement actually strengthens protection without making the assistant unnecessarily restrictive.

Finally, we deployed the application with an interactive dashboard so users can explore the security workflow and inspect evaluation results.

Live Demo: https://aiwatchtower.onrender.com/

Challenges we ran into

Accomplishments that we're proud of

  1. Balancing security and usability A policy that blocks every action may prevent attacks, but it also makes an AI assistant useless. I needed to evaluate security improvements alongside their impact on legitimate tasks. Replay-based testing helped us compare both outcomes.

  2. Preventing unsafe AI-generated patches AI-generated changes can introduce vulnerabilities or weaken existing restrictions. I addressed this by separating patch generation from deterministic validation and human approval.

  3. Evaluating unseen threats Passing a known attack scenario does not guarantee protection against new attacks. I recognized the importance of testing against independent scenarios and interpreting results cautiously, especially when evaluation datasets are small.

  4. Making security decisions auditable Security decisions need to be understandable and reviewable. I worked to expose policy decisions, proposed changes, and evaluation evidence through the dashboard rather than relying solely on AI-generated explanations.

What we learned

Building AI Watchtower taught me that AI security requires more than a powerful model or a collection of blocking rules.

I learned why deterministic enforcement is important when protecting sensitive operations, why AI-generated changes must be validated, and why every security improvement should be tested against both malicious and legitimate scenarios.

I also learned that evaluation metrics need context. Results from a limited set of simulated attacks can help identify weaknesses, but they cannot prove that a system is secure against every possible threat.

Most importantly, I learned that human oversight should be built into the architecture, not added as an afterthought.

What's next for AI Watchtower

I plan to strengthen AI Watchtower by:

Expanding the dataset with more diverse prompt-injection and tool-misuse scenarios.

Testing against independently authored and previously unseen attacks. Improving policy validation and reducing false positives. Evaluating security performance across more models and attack strategies. Integrating with additional AI agents and tools in controlled environments. Improving audit trails, policy versioning, and explanations for security decisions. Developing stronger benchmarks to measure how well policy improvements generalize to new threats.

My long-term vision is to build an adaptive security layer that helps AI agents become safer over time while keeping critical security decisions transparent, testable, and under human control.

Built With

  • access
  • ai
  • api
  • automated
  • control
  • cybersecurity
  • defense
  • fastapi
  • injection
  • javascript
  • language
  • large
  • models
  • nemotron
  • nvidia
  • policy-based
  • prompt
  • python
  • react
  • rest
  • safety
  • security
  • testing
Share this project:

Updates

Submission history