Apertus Invoice Assistant

Autonomous AI agent that reconciles supplier invoices and detects fraud — powered by Apertus, Switzerland's sovereign LLM.

Comment

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for Apertus Invoice Assistant

💡 Inspiration

Every company receives hundreds of supplier invoices every month. Each one must be manually compared against its purchase order to catch amount mismatches, VAT errors, and fraudulent invoices. This process is time-consuming, error-prone, and rarely catches sophisticated fraud.

We built Apertus Invoice Assistant to automate this workflow using Apertus, Switzerland's sovereign LLM. Our goal: give small and medium businesses access to a trustworthy, sovereign, and affordable fraud detection tool — one that keeps their financial data in-house.

🚀 What it does

Apertus Invoice Assistant is an autonomous AI agent that:

  1. Extracts structured data from an invoice (text, image, or PDF)
  2. Finds the matching purchase order in the database
  3. Compares amounts, VAT, and line items
  4. Detects fraud using 11+ heuristic rules, statistical analysis (z-score), and machine learning (Isolation Forest)
  5. Generates a professional PDF report with a risk score (0-100) and detailed anomalies
  6. Decides: sends an alert OR marks the invoice as approved

The app supports 5 languages via OCR (French, English, German, Italian, Spanish), handles PDF and image uploads, and runs on a dual-mode LLM architecture:

  • ☁️ Cloud mode: Groq (openai/gpt-oss-120b) for speed
  • 🏠 Sovereign mode: Apertus 8B via Ollama, 100% local, no internet required

🛠️ How we built it

Component Technology
LLM (cloud) Groq API — openai/gpt-oss-120b
LLM (local) Apertus 8B via Ollama (Q4_K_M)
Backend FastAPI + Python 3.11
Fraud detection 11 heuristic rules + z-score + scikit-learn Isolation Forest
OCR Tesseract 5.5 (5 languages) + Poppler for PDF
Database SQLite (suppliers, PO history, invoice history)
Frontend Vanilla HTML / CSS / JS + Chart.js dashboard
PDF reports reportlab
Deployment Docker + Render
Tests 25 unit tests (pytest) + CI/CD GitHub Actions

The agent uses native tool calling with 6 tools chained autonomously: extract_generic → find_purchase_order → compare_amounts → detect_fraud → send_alert / mark_as_approved

🧗 Challenges we faced

1. Making the agent follow the full workflow. Early versions of the agent would stop after extraction without calling detect_fraud. We solved this with a strict system prompt and an automatic nudge mechanism: if the agent tries to finish without calling detect_fraud, the code sends a reminder and continues the loop.

2. Multilingual OCR accuracy. Invoices come in many formats and languages. We combined Tesseract with 5 language packs and a regex-based pre-processor, achieving reliable extraction on French, English, German, Italian and Spanish documents.

3. Deploying system dependencies. Tesseract and Poppler are system packages, not Python ones. On a Python-native platform, apt-get install fails because of the read-only file system. We solved this by packaging everything into a Dockerfile that installs Tesseract, Poppler, and the 5 language packs at build time.

4. Fraud detection without false positives. A naive rule engine flags too many invoices. We weighted each rule and combined heuristics with ML-based anomaly detection (Isolation Forest), producing a nuanced risk score instead of a binary flag.

5. Sovereign deployment. We wanted a version that runs 100% locally, without any cloud dependency. We implemented a dual-mode client that switches between Groq (cloud) and Ollama (local Apertus) via a single environment variable.

🏆 Accomplishments

  • ✅ 25 unit tests, 100% pass rate, with CI/CD via GitHub Actions
  • ✅ Dual-mode LLM: 100% sovereign local mode with Apertus 8B
  • ✅ Advanced fraud detection (11 rules + ML) with explainable anomalies
  • ✅ Multilingual OCR (5 languages)
  • ✅ Professional PDF reports with risk scores and detailed anomalies
  • ✅ Live dashboard with real-time stats
  • ✅ Fully deployed and accessible online
  • ✅ Open source (MIT License)

📚 What we learned

  • How to build an autonomous AI agent with native tool calling
  • How to combine heuristic rules with machine learning for fraud detection
  • How to handle OCR and PDF processing in a production pipeline
  • How to deploy a Python app with system dependencies using Docker
  • The importance and feasibility of sovereign AI infrastructure

🔮 What's next for Apertus Invoice Assistant

  • Fine-tune Apertus on real invoice datasets for even better extraction
  • Multi-currency support with automatic conversion
  • Accounting software integrations (SAP, QuickBooks, Odoo)
  • Mobile app (React Native)
  • Blockchain-based audit trail for regulatory compliance
  • Federated learning across companies (privacy-preserving)

🔗 Links

👤 Author

Anio Joseph — Built for Hack Apertus, October 2026 (Track 2B: Apertus Adoption — Own Project)

Built With

Share this project:

Updates

Submission history