Legal Auditor - AI Compliance Agent

AI agent that detects PII leaks & regulatory violations in real-time. Uses Elasticsearch hybrid search to match documents against GDPR/HIPAA policies and auto-flag compliance risks.

Comment

Inspiration

Compliance teams face an impossible task: manually reviewing every Slack message, email, and document to check for PII leaks or regulatory violations. A single missed Social Security Number in a support ticket can result in GDPR fines up to €20 million. I wanted to build an agent that turns hours of manual review into seconds of automated analysis.

What it does

Legal Auditor is a multi-step AI agent that:

  1. Ingests documents in real-time through a PII detection pipeline
  2. Identifies sensitive data (SSNs, credit cards, emails, phone numbers)
  3. Matches content against compliance policies using Elasticsearch Hybrid Search (BM25 + Vector)
  4. Flags violations with specific policy citations and risk scores
  5. Provides a real-time dashboard for compliance officers

How I built it

  • Elasticsearch Serverless for data storage and hybrid search
  • Ingest Pipelines with custom PII detection logic
  • Hybrid Search combining keyword (BM25) and semantic vector search for accurate policy matching
  • Node.js + Express API server
  • Vanilla JavaScript frontend for a lightweight, fast dashboard

The agent monitors three Elasticsearch indices:

  • compliance-policies - GDPR, HIPAA, and internal policies
  • monitored-documents - Incoming documents with PII annotations
  • flagged-violations - High-risk items requiring review

Challenges I faced

  1. Elasticsearch Serverless limits - The strict regex limits on Painless scripts forced me to move PII detection from ingest pipelines to the application layer. This actually improved performance!

  2. Hybrid search tuning - Balancing keyword relevance vs. semantic similarity required experimentation with boost values and reciprocal rank fusion.

  3. Risk scoring algorithm - Designing a scoring system that accurately prioritizes violations based on PII type, policy severity, and context.

What I learned

  • Elasticsearch's hybrid search is incredibly powerful for matching unstructured text against policy databases
  • Moving compute-heavy operations (like regex-based PII detection) to the application layer can actually be more scalable than doing it in ES
  • Agent Builder's custom tools make it easy to connect LLMs with Elasticsearch data

What's next

  • Integration with Slack/Teams for real-time alerts
  • Support for more compliance frameworks (SOC2, PCI-DSS)
  • Remediation workflow automation (auto-redact, auto-ticket creation)

Built With

Share this project:

Updates

Submission history