InvoSense

Inspiration Accounts payable teams still process most invoices by hand—opening emails, retyping vendor and GST details, and checking tax rules before any payment is released. We built InvoSense to show that this entire flow can run autonomously: from an unread Gmail attachment to a compliance-checked, policy-driven payment decision with a full audit trail.

What it does InvoSense watches Gmail for unread messages with invoice attachments, extracts structured fields (vendor, GST ID, totals, dates) with confidence scores and bounding boxes, and runs four compliance checks against ClickHouse—vendor verification, Malaysia tax-rate rules, totals validation, and duplicate detection. Clean invoices are auto-approved and paid via x402 USDC on Base Sepolia, with proof published to cited.md. Rejected invoices surface clear reasons on a live dashboard. Finance teams see every step in real time: Received → Extracting → Compliance → Decision → Payment.

How we built it The Python/FastAPI backend handles ingest, extraction, and compliance. Composio polls Gmail and downloads attachments. TrueFoundry powers vision-based extraction; Tesseract draws field bounding boxes. ClickHouse Cloud stores vendor master data, tax rules (from cited Malaysia GST policy), invoice history, and policy config. A Node.js/Express frontend syncs with the backend, displays pipeline progress, triggers x402 payments through viem, and publishes settlement records. Malaysian demo invoices and synthetic rejection scenarios validate both approve and reject paths end to end.

Challenges we ran into ClickHouse Cloud timeouts during compliance caused invoices to stall mid-pipeline—we added retries, longer timeouts, and a retry endpoint. The frontend and backend used different data models, so we built a mapping layer to unify pipeline status, decisions, and payment state. Gmail only picks up unread emails with attachments, which tripped up early testing. Bounding boxes existed in extraction JSON but weren’t visible until we overlaid SVG boxes on the invoice image in the UI.

Accomplishments that we're proud of A fully working email-to-payment demo: send an invoice by Gmail, watch it process on the dashboard, and see auto-approve with an on-chain tx hash—or rejection with a detailed compliance reason. All 10 hackathon demo invoices pass compliance; synthetic scenarios correctly flag tax, vendor, totals, and duplicate issues. The pipeline is policy-driven, auditable, and built on real integrations—not mocks alone.

What we learned Autonomous AP needs more than OCR—it needs compliance as a first-class step between extraction and payment. Separating the Python processing spine from the Node payment/UI layer kept concerns clean but required careful contract design between services. Malaysia GST rules change by date, so tax logic must be data-driven (ClickHouse + cited policy), not hardcoded. Real-world cloud latency (ClickHouse, Gmail polling) matters as much as model accuracy for a demo that feels production-ready.

What's next for InvoSense Production Gmail triggers instead of polling, PDF extraction support, human-in-the-loop review for borderline cases, mainnet x402 with real FX, ERP export (SAP/QuickBooks), and persistent invoice storage beyond in-memory API state. We’d also add webhook notifications and a compliance analytics view so finance teams can track rejection trends and vendor risk over time.

Built With

• clickhouse
• composio
• crewai
• openui
• truefoundry
• x402