• 

  web page

Project Story

Inspiration

In today's rapidly evolving digital landscape, businesses increasingly rely on LLMs to draft critical documents—from financial reports and legal contracts to strategic plans and compliance filings. However, we noticed a dangerous gap: there was no automated way to ensure that AI-generated content remained faithful to approved source material. A single undetected contradiction in a contract clause or financial projection could lead to costly errors, compliance violations, or strategic missteps. We built AuditFlow to fill this critical oversight—creating a "safety net" for human-LLM collaboration.

What It Does

AuditFlow automatically compares LLM-generated outputs against trusted reference documents (the "source of truth"), detecting:

• Factual contradictions (e.g., altered numbers, changed timelines, modified terms)
• Statistical drift using Jensen-Shannon divergence
• Semantic deviations (shifts in meaning or intent)
• Novel, unsupported claims

When risks exceed configurable thresholds, the system triggers tailored escalation protocols—from simple warnings to immediate human review—complete with actionable checklists and detailed audit trails.

How We Built It

• Frontend: Next.js + React for a dynamic, bilingual (EN/ES) interface
• Backend: Node.js/Express REST API hosted on Railway
• AI Integration: Google Gemini API for deep semantic analysis
• Core Algorithms: Custom JSD (Jensen-Shannon Divergence) implementation for quantitative drift detection
  [ JSD(P \parallel Q) = \frac{1}{2} D_{KL}(P \parallel M) + \frac{1}{2} D_{KL}(Q \parallel M) ] where (M = \frac{1}{2}(P + Q))
• Rule Engine: Configurable escalation rules based on severity scores
• Infrastructure: Vercel (frontend), Railway (backend), environment-based configuration

Challenges We Ran Into

• Backend-Frontend Integration: CORS issues and double-slash URL problems that broke API calls
• Gemini API Latency: Balancing thorough analysis with real-time responsiveness
• Multilingual Support: Ensuring consistent UX and terminology across Spanish and English interfaces
• Threshold Calibration: Tuning JSD and severity thresholds to minimize false positives/negatives
• State Management: Handling complex audit state across demo and custom input modes

Accomplishments We're Proud Of

• Creating a fully functional, dual-language audit system in a short timeframe
• Successfully integrating statistical methods (JSD) with LLM-powered semantic analysis
• Building an intuitive UI that clearly communicates risk levels and required actions
• Developing a scalable rule engine that supports multiple risk domains
• Achieving seamless deployment across Vercel and Railway

What We Learned

• The importance of robust error handling in full-stack applications
• How to effectively combine statistical analysis with modern LLM APIs
• The nuances of designing escalation workflows for different risk profiles
• Practical considerations for deploying and connecting separate frontend/backend services
• How to communicate complex audit results in an accessible, actionable way

What's Next for AuditFlow

• Real-Time API Monitoring: Continuous auditing of LLM endpoints
• Collaborative Review Interface: Multi-user workflow for team-based audit resolution
• Advanced NLP Features: Named entity recognition for specific compliance tracking
• Custom Rule Builder: Drag-and-drop interface for non-technical users to create escalation rules
• Audit Trail Analytics: Dashboards showing drift trends and common contradiction types
• Integration Marketplace: Connectors for popular tools (Slack, Jira, Microsoft 365)
• Mobile Application: On-the-go audit review and approval system

AuditFlow represents our vision for responsible AI adoption—where human expertise and artificial intelligence collaborate with clarity, accountability, and built-in safeguards.

Built With

• express.js
• html5/css3
• javascript
• next.js
• node.js
• react
• tailwind