AuditPilot — Agentic Compliance Audit Workflow

AuditPilot automates compliance audits by comparing policy documents with checklists, detecting gaps, routing risks for human review, and generating executive reports.

Comment

Inspiration

Compliance audits are important, but they are often slow, manual, and repetitive. Reviewers need to read long policy documents, compare them against checklist requirements, identify missing controls, estimate risk, assign owners, and prepare reports for management.

I wanted to build a tool that makes this process faster without removing human accountability. Audit findings can affect real security and business decisions, so the system should help the reviewer, not replace them.

That idea became AuditPilot: an agentic compliance audit workflow that automates the heavy analysis while keeping a human in control before the final report is generated.

What it does

AuditPilot allows a reviewer to upload an internal policy document and an audit checklist. From there, a LangGraph-based multi-agent workflow runs the audit process.

The workflow:

  • Extracts policy controls from the uploaded policy document
  • Extracts audit requirements from the checklist
  • Generates embeddings for policy controls
  • Stores and retrieves controls using MongoDB Atlas
  • Performs semantic matching between requirements and policy evidence
  • Verifies whether each requirement is compliant, partially compliant, or non-compliant
  • Detects compliance gaps
  • Scores each gap by risk level
  • Routes findings to responsible owners
  • Pauses at a Human-in-the-Loop review checkpoint
  • Generates a structured executive audit report after reviewer decisions are completed

The final report includes an executive summary, compliance gaps, risk severity, mapped evidence, reviewer decisions, owners, and recommended remediation actions.

How I built it

AuditPilot is built as a full-stack web application using React, TypeScript, Node.js, and Express, with the help of the wonderful tool Google AI Studio. The backend uses a LangGraph-based workflow to coordinate specialized audit agents. Each node in the graph handles a specific step, including document ingestion, control extraction, checklist extraction, embedding and storage, vector retrieval, compliance verification, risk scoring, owner routing, human review, and report generation.

Gemini is used for reasoning-heavy audit tasks such as extracting controls, verifying compliance gaps, scoring risk, assigning owners, and generating the final executive report.

MongoDB Atlas is used to store policy controls and vector embeddings so checklist requirements can be matched against semantically relevant evidence.

The frontend provides a guided audit workspace where users can upload documents, run the audit pipeline, review findings, approve or accept risks, and view the final generated report.

Challenges I ran into

One major challenge was designing the workflow so it felt like a real audit process instead of a simple chatbot response. Compliance reviews need traceability, structured findings, risk ownership, and human approval.

Another challenge was handling quota limits and API reliability during demos. Since Gemini calls can be rate-limited, I added fallback and sample-demo behavior so the project can still be demonstrated reliably with sample documents.

The Human-in-the-Loop stage was also important. I had to make sure the workflow paused before report generation and only resumed after all findings were reviewed.

What I learned

I learned how to design an AI system as a coordinated workflow instead of a single prompt. LangGraph made it easier to model the audit process as a series of specialized agents with clear responsibilities.

I also learned how important human review is in compliance and security workflows. AI can accelerate analysis, but the final decision still needs context, judgment, and accountability from a reviewer.

Accomplishments that I'm proud of

I am proud that AuditPilot is more than a document summarizer. It performs a full compliance workflow from uploaded documents to human-reviewed findings and a final executive report.

I am also proud of the Human-in-the-Loop design, because it makes the system safer and more realistic for compliance use cases.

What's next for AuditPilot

Next, I would like to add support for more compliance frameworks such as SOC 2, HIPAA, PCI DSS, and NIST. I would also like to add persistent user accounts, audit history, reviewer comments, exportable PDF reports, and deeper integrations with ticketing tools like Jira or ServiceNow.

Long term, AuditPilot could become a compliance operations assistant that continuously monitors policy gaps, assigns remediation tasks, and helps teams prepare for audits before deadlines.

Built With

Share this project:

Updates