ConsentOps Agent

A Gemini + Fivetran agent that finds where personal data spread, prepares safe cleanup actions, requires human approval, and produces an audit report.

Comment

Inspiration

When someone withdraws consent, their data rarely sits in one table. It spreads across CRM, commerce, support, marketing, analytics, and payments—often landed in a warehouse by Fivetran. Most demos stop at policy slides. I built ConsentOps Agent to model the full operational loop: discover → plan → human approve → execute → live re-scan → audit.

What it does

ConsentOps Agent helps operators respond to a consent withdrawal for a synthetic subject (Ana Reyes):

  1. Scan — Read-only Fivetran MCP discovery (connections, sync state, destinations) plus a BigQuery warehouse scan across seven demo tables (37 matches when fully seeded).
  2. Plan — Gemini (or deterministic fallback) classifies every record: delete, anonymize, retain (with reason), or review.
  3. Approve — Reviewer selects specific action IDs (e.g. one delete, one anonymize, one retain)—not approve-all.
  4. Execute — Only approved actions run against BigQuery; payment deletes are blocked by policy.
  5. Verify — Live re-scan (e.g. 37 → 35 after one delete + one anonymize; retain stays).
  6. Audit — JSON + markdown report of connectors inspected, actions run, and records remaining.

Synthetic demo data only. Not legal advice or compliance certification.

How I built it

  • Next.js / TypeScript dashboard with a five-phase workflow (Scan → Plan → Approve → Execute → Audit).
  • Fivetran MCP (read-only, FIVETRAN_ALLOW_WRITES=false) via Cloud Run runtime + REST fallback for connector status.
  • BigQuery synthetic warehouse (npm run bigquery:setup) with scan / execute / verify modes.
  • Gemini cleanup planner with deterministic safety validation and fallback.
  • Agent APIs (POST /api/agent/scan, /plan, /fivetran) for Google Cloud Agent Builder / ADK integration; optional Vertex AI Agent Engine deploy.
  • 159 Vitest tests covering approval gates, classification rules, and public API sanitization.

Challenges I ran into

  • Wiring Fivetran MCP on Cloud Run while keeping the demo read-only and safe for public judges.
  • Aligning live BigQuery match counts with fixture data (re-seed via bigquery:setup).
  • Redacting raw Fivetran connection/schema IDs on public agent endpoints without breaking internal discovery.
  • Keeping human-in-the-loop UX honest—the approval slider only completes after successful execution.

Accomplishments that I'm proud of

  • End-to-end multi-step agent (not chat-only): MCP discovery → warehouse scan → Gemini plan → explicit approval → DML execute → live verify → audit.
  • Safety layer: no table-wide deletes, payment protection, plan binding, demo-mode subject allowlist.
  • Fivetran + BigQuery narrative: pipeline lineage on scan links ingestion to the warehouse ConsentOps cleans up.
  • Production-minded Cloud Run deployment with OpenAPI spec and Agent Builder playbook.

What I learned

  • Fivetran MCP is strongest for agent tool-calling at scan time; REST is a good read-only fallback for status panels.
  • Judges care about live proof (match counts, before/after re-scan) more than architecture diagrams.
  • Gemini plans must pass deterministic validation before any destructive work is offered to a human reviewer.

What's next for ConsentOps Agent

  • Connector-scoped retention policies in the audit narrative.
  • Optional Agent Engine playground deep-link from the dashboard.
  • Broader warehouse adapters (still synthetic-only for demos).

Try the hosted demo: https://consentops-agent-538209538110.us-central1.run.app/

Built With

  • bigquery
  • docker
  • fivetran-mcp-server
  • fivetran-rest-api
  • gemini-api
  • google-adk
  • google-cloud-run
  • model-context-protocol-(mcp)
  • next.js
  • node.js
  • react
  • tailwind-css
  • terraform
  • typescript
  • vertex-ai-agent-engine
  • vitest
  • zod
Share this project:

Updates