TriBIT's Geo-Compliance Detector

Automated compliance — precise, transparent, and scalable.

Comment

From Guesswork to Governance: TriBIT’s Geo Compliance Detector

Inspiration

The inspiration for “From Guesswork to Governance” came from the overwhelming complexity companies face when launching global features. Each jurisdiction (EU, US federal, California SB 976, Utah, Florida, etc.) has unique compliance obligations, but traditional processes rely on manual legal reviews. These are slow, inconsistent, and prone to error — creating bottlenecks and risking costly violations.

We saw an opportunity to leverage cutting-edge AI to transform compliance from reactive guesswork into proactive, automated governance. The vision was clear: a system that can instantly analyze any product feature against a comprehensive legal knowledge base, flag geo-specific compliance requirements, and output audit-ready assessments with full provenance.

What It Does

Our system automates geo-specific compliance detection by analyzing feature artifacts (titles, descriptions, PRDs, TRDs) against an indexed legal knowledge base.

Users input a feature (e.g., “age verification for social media”). The system:

  • Auto-detects jurisdictions using regex + semantic heuristics to pick up compliance cues (e.g., ASL, GH, Utah, EU, parental consent).
  • Retrieves relevant legal documents via hybrid search (dense semantic embeddings + sparse keyword matching).
  • Reranks candidates with a cross-encoder to ensure the best law snippets rise to the top.
  • Classifies the feature using Groq’s Llama-3.1-8B-Instant, which outputs structured JSON: a Yes/No/Unclear verdict, reasoning, confidence, and linked regulations.
  • Produces audit-ready provenance, logging rule hits, inferred regions, retrieval parameters, rerank method, elapsed time, and request IDs.
  • Scales to batch processing, so enterprises can classify entire release trains in one run.

Coverage: US Federal law, state-level laws (CA, UT, FL), and EU regulations. The KB is expandable with live law uploads.

How We Built It

AI/ML Pipeline

  • Embeddings: We use BGE-M3 embeddings (1024-dim) for dense semantic retrieval. Sparse embeddings complement this for keyword sensitivity.
  • Vector DB: Qdrant provides hybrid dense+sparse search, region-aware filtering, and fallback strategies.
  • Reranker: A cross-encoder (MS MARCO MiniLM-L-6-v2) reranks candidate chunks. If it fails, we fall back to lexical scoring.
  • LLM: Groq’s Llama-3.1-8B-Instant runs in strict JSON schema mode, producing deterministic outputs (yes/no/unclear + reasoning + confidence).

Mathematically, hybrid retrieval combines cosine similarity (\(\cos(\theta)\) in embedding space) with BM25 lexical scores:

$$ \text{score}(q, d) = \alpha \cdot \cos(E_q, E_d) + (1 - \alpha) \cdot \text{BM25}(q, d) $$

Backend Infrastructure

  • FastAPI with REST endpoints for classification, search, feedback.
  • LangChain orchestrates chains for retrieval + LLM + rerank.
  • Chunking: Header-aware splitting + recursive fallback ensures long laws remain coherent chunks.

Frontend Experience

  • Next.js 14 + TypeScript for a robust UI.
  • Tailwind CSS for responsive styling.
  • Features: Analyze (single artifact), Batch (CSV classification), Laws (upload/search), Provenance (audit trail).

Data Processing

  • Markdown-aware ingestion pipeline with header mapping.
  • Metadata enrichment for region tagging.
  • Hybrid retrieval tuned with k, MMR diversity, and jurisdiction filters.

Challenges We Faced

  • Multi-service orchestration: Dockerizing Qdrant, Python backend, Node frontend.
  • Legal document chunking: Preserving semantic boundaries while handling long laws.
  • Retrieval quality: Balancing dense embeddings vs. sparse keywords for high recall and precision.
  • LLM reliability: Preventing malformed outputs with JSON schema + strict prompts.
  • Latency: Optimizing embedding generation, reranking, and Groq inference to stay <2s per request.

Accomplishments

  • Sophisticated RAG architecture: Dense + sparse hybrid retrieval with reranking.
  • Audit-ready provenance: Every decision is explainable and reproducible.
  • Feedback loop with few-shot memory: Reviewer corrections feed back into the system, creating institutional memory.
  • Batch compliance reviews: Days of manual work reduced to minutes.
  • Cross-jurisdiction coverage: Indexed US federal, multiple US states, and EU laws into one KB.

What We Learned

  • Domain specialization matters: Generic embeddings underperform. Legal cues, regex patterns, and region inference are crucial.
  • Prompt design = architecture: JSON schema enforcement and confidence calibration make LLMs reliable.
  • Fallbacks are essential: Rerankers may fail; dense retrieval may miss exact terms. Multiple strategies ensure robustness.
  • Explainability is non-negotiable: Compliance requires full provenance — rule hits, law text, request IDs.
  • Feedback creates few-shot memory: Human corrections bootstrap a growing internal dataset. Over time, this yields better heuristics and higher precision without retraining.

What’s Next

  • Advanced fine-tuning: Train legal-domain embeddings and cross-encoders for even higher retrieval precision.
  • Multilingual support: Index EU laws in native languages, requiring multilingual embeddings.
  • Automated legal updates: Continuous ingestion of new/changed laws.
  • Compliance dashboards: Risk scoring, audit trails, automated reporting.
  • Collaborative workflows: Multi-user validation and correction integrated into the feedback loop.

  • Alternative detection mechanisms: Beyond text artifacts, extend to:

    • Static code analysis (region gates, geo-handlers).
    • Runtime analysis (telemetry signals).
    • Data flow tracing (sensitive data handling).

How It Meets Hackathon Goals

  • Boosting LLM Precision:

    • Hybrid retrieval (dense + sparse).
    • Cross-encoder reranking.
    • Regex-based rule detection.
    • Few-shot memory from feedback.

  • Full Automation:

    • Self-improving via feedback loops.
    • JSON schema prevents drift.
    • Batch mode scales to enterprise use.

  • Alternative Detection Mechanisms:

    • Future work on static code analysis, runtime telemetry, and data tracing.

Task requirements addressed:

  • Reduce Compliance Governance Costs: Batch triage replaces manual review.
  • Mitigate Regulatory Exposure: Region-aware retrieval lowers false negatives.
  • Enable Audit-Ready Transparency: Provenance captures every step, with unique request IDs.

Built With

Share this project:

Updates

Submission history