AI Governance-Cockpit

A live assurance layer that flags biased AI outputs in real time, matches them to real governance controls by meaning, and writes an auditable trail.

Comment

Inspiration

AI teams ship new features daily, but the people accountable for governing that AI compliance and risk officers are stuck working retrospectively, relying on spreadsheets and after-the-fact audits. We wanted to close that gap. AI bias in hiring, lending, and healthcare isn't a hypothetical risk; it's something compliance teams face today, and it's heavily regulated. We chose this problem because it's real, immediately legible to anyone, and because one of us (Safi) has a background in AI governance frameworks like NIST AI RMF and ISO 42001 which meant we could build something credible, grounded in real controls, instead of a toy demo.

What it does

The AI Governance Cockpit is a live assurance layer that sits between a company's AI systems and the people responsible for governing them. You paste any AI-generated output for example, a résumé screener's justification for a hiring decision and the Cockpit:

  1. Flags it if it references protected characteristics or otherwise violates a governance control.
  2. Semantically matches the violation to the specific governing control using vector similarity search by meaning, not keyword matching.
  3. Writes an immutable audit record with a UUID and timestamp, giving the compliance officer a full traceback: output → violation → matched control → audit entry, exportable as evidence.

We tested it with two contrasting inputs a clean, job-relevant assessment and one with hidden bias (referencing a candidate's age and maternity leave) to prove the system distinguishes real violations from normal text rather than flagging everything indiscriminately.

How we built it

  • Amazon Aurora PostgreSQL 17 (Serverless v2) as our system of record storing the control library, audit events, violations, and a risk register roll-up table.
  • pgvector with an ivfflat index and cosine similarity for semantic search. We embedded a library of 12 verified governance controls drawn from NIST AI RMF, ISO 42001, and anti-discrimination/fairness law.
  • Voyage AI embeddings (512-dimensional, voyage-3-lite) to generate vector representations of both the control descriptions and incoming AI outputs, so matching happens on meaning rather than surface-level keywords.
  • Next.js on Vercel for the frontend and API layer the hero interaction is paste → flag → traceback, designed to make the backend's intelligence visible and tangible rather than hidden behind an opaque "yes/no" result.
  • Prisma as our ORM layer for the relational tables, with raw SQL for the vector operations pgvector requires.

We divided work along a "pipe and brain" split: one of us built the Next.js frontend, Aurora wiring, and Vercel deployment; the other built the control library, violation detection rules, and the pgvector embedding layer. Locking the Flag API contract on day one a simple JSON shape for requests and responses let both tracks move in parallel without blocking each other.

Challenges we ran into

  • Indirect, coded language slipped past our first version. Our original keyword-only approach caught explicit terms like "age" or "maternity leave," but missed indirect phrasing like "energetic recent graduates without family obligations." We fixed this by enriching our control descriptions with examples of coded language so the embeddings had better anchor points, and by running the pgvector similarity search unconditionally on every input rather than gating it behind a keyword hit so semantic matching catches what keyword rules miss.
  • Database authentication detours. We hit IAM-only authentication locks on our first Aurora cluster (caused by Internet Access Gateway being enabled in Express Create mode), which forced us to recreate the cluster with Standard Create and self-managed credentials.
  • SSL and connection quirks between Vercel and Aurora. Vercel's serverless functions run from dynamic IPs, so we had to open our security group broadly and explicitly disable strict SSL verification (NODE_TLS_REJECT_UNAUTHORIZED=0) for the hackathon environment.
  • Embedding provider rate limits. We started with OpenAI but hit billing and quota issues, switched to Voyage AI, and then had to throttle our seed script to stay under their free-tier rate limit while embedding our control library.
  • Vector dimension mismatches. Our vector(1536) column didn't match Voyage's actual 512-dimensional output, which we only caught once Postgres rejected the insert a good reminder to verify embedding dimensions against the model's actual spec, not assumptions carried over from a different provider.

What we learned

Real semantic search requires real anchor quality embeddings are only as good as the text you embed them from, and adding concrete examples of coded/indirect language to our control descriptions made a measurable difference in catching what keyword rules alone would miss. We also learned that locking a simple API contract early is what actually lets a small team move fast in parallel without stepping on each other.

Built With

Share this project:

Updates