AgentProtector

Inspiration

What inspired this project

AI agents are getting powerful fast. They can read emails, access files, use tools, and even take actions on behalf of users. But one big problem keeps coming up: agents can be tricked (prompt injection), or they can accidentally access too much sensitive data (PII, financial data, internal documents).

We built AgentProtector because we wanted a security and governance layer that sits between any agent and any tool. The idea is simple: No tool access unless the request is checked, logged, and either automatically allowed, denied, or approved by a human.

What it does

What we built

AgentProtector is an Agent Access Gateway that does:

Intercepts agent requests before they reach tools (Gmail, Drive, database, etc.)

Applies hard policy rules (like “bulk export of PII is denied”)

Uses Gemini to score risk, detect suspicious prompts, and provide explainability

Routes uncertain cases to a Manager Console for approval (human-in-the-loop)

Creates audit logs so every decision is traceable

It includes a Demo Agent UI so judges can test the whole workflow with only clicks.

How we built it

Agent submits a request (purpose + tool + scope + data type)

Gateway checks hard rules and calls Gemini for risk + suspicious detection

Gateway stores the decision + full policy JSON in DB

If NEEDS_APPROVAL, the manager approves/denies in Manager Console

If approved, the agent can execute the tool action

Everything is logged to an audit trail

Challenges we ran into

Storing policy evidence (what rules fired + what Gemini recommended) is important for audits

How we built it (architecture + flow)

Agent submits a request (purpose + tool + scope + data type)

Gateway checks hard rules and calls Gemini for risk + suspicious detection

Gateway stores the decision + full policy JSON in DB

If NEEDS_APPROVAL, the manager approves/denies in Manager Console

If approved, the agent can execute the tool action

Everything is logged to an audit trail

Challenges we faced

Judge-friendly setup: Hackathons require judges to run/test quickly. So we built a DB fallback that automatically uses SQLite when Postgres is not configured.

Portability of JSON storage: SQLite and Postgres behave differently. We made policy storage portable using SQLAlchemy JSON with Postgres JSONB variant.

Prompt injection detection: It’s not enough to rely on the model. We combined rules + Gemini signals and stored the reasoning for transparency.

UX clarity: The hardest part was making it easy to understand in 2 minutes. We focused on clickable test cases + manager approval flow.

Accomplishments that we're proud of

What we learned

What we learned

While building this, we learned:

Security for AI agents is not only about “blocking” — it’s about safe workflows

“Human-in-the-loop” can be fast and practical when the UI is designed well

Model reasoning needs to be visible (explainability), otherwise approvals feel blind

Storing policy evidence (what rules fired + what Gemini recommended) is important for audits

What's next for AgentProtector

Built With

• dotenv
• fastapi
• gemini-api-key
• jinja
• postgresql
• python
• sqlalchemy
• sqlite
• uvicorn