duecollect

Inspiration

Every small business has the same awkward job: chasing people who owe them money. You write polite reminders, lose track of who owes what, and then a payment screenshot lands in your inbox and you have no idea if it's real. It eats hours every week, and the wrong tone can damage a relationship. We wanted to build something that does the chasing and the checking, so the owner doesn't have to.

What it does

DueCollect turns collections into something that runs itself.

1. The admin uploads their payment QR code once. The app decodes it and learns which account money should arrive in.
2. An AI looks at everything each customer owes, past dues and upcoming ones, and writes a single email with the right tone: friendly if they're on time, firmer if they're weeks late. The QR is attached.
3. The customer replies with a screenshot of their payment.
4. The AI reads the screenshot, then plain code verifies it. If every check passes, the invoice is marked paid. If anything looks off, it's flagged for the admin with the reasons and the image.
5. Everything is logged to CSV files (debts.csv, payments.csv, emails.csv) that open straight in Excel.

How we built it

• Next.js + TypeScript for the dashboard, admin panel, login screen and API routes.
• NVIDIA Cosmos3 served through vLLM as the local AI. It exposes an OpenAI-compatible API with image input, so one client handles both writing emails and reading bills. Any OpenAI-compatible endpoint works too.
• jsQR + sharp to decode the uploaded QR and read the receiving UPI ID.
• nodemailer to send emails, imapflow + mailparser to read customer replies from the mailbox.

The key design decision: the AI only reads evidence; it never decides. The model extracts the amount, payee, transaction ID and any signs of tampering. Then deterministic code makes the call. A bill is accepted only if the payment succeeded, the amount matches, the money went to our QR's account, the transaction ID has never been used, nothing looks edited, and the AI is confident enough

and settle exactly those invoices. Customers usually clear their oldest dues first, so this matches real behavior.

Challenges we ran into

• Trusting AI output. Letting a model say "this is paid" is dangerous, since an email can contain instructions aimed at the model. Splitting reading (AI) from deciding (code) solved that.
• A bug our tests caught. If a customer's first screenshot was blurry and got parked for review, their clearer second attempt was rejected as a "duplicate" of their own pending report. We changed the rule so only confirmed payments, or reports against someone else's invoices, count as duplicates, and a verified retry now upgrades the pending record.
• A security hole. The original dashboard looked private, but its API routes were open to anyone with the URL. We added a proper admin login and locked every route behind it, plus brute-force protection.
• No GPU to test on. We tested the whole pipeline against a mock of the model server and a mock mail server, which covered 25 end-to-end checks, but the real Cosmos model needs an NVIDIA GPU.

What we learned

• Constrain the AI to extraction and keep decisions in code you can test.
• Mock servers let you test an AI pipeline end to end without the model.
• A login screen that only hides the page protects nothing; the API routes have to be locked too.

Honest limitations

A well-made fake screenshot could still pass these checks. For large amounts, confirming against a bank statement is still the right call. PDFs are saved for review but not read by the AI, and typed claims without a screenshot always go to manual review.

What's next

• Read PDF receipts
• Check payments against bank statements or the UPI provider
• WhatsApp and SMS reminders
• A hosted-model fallback for machines without a GPU

Built with: Next.js, TypeScript, NVIDIA Cosmos3, vLLM, nodemailer, imapflow, mailparser, sharp, jsQR

Built With

• computer-vision
• csv
• date-fns
• imap
• jsqr
• llm
• nextjs
• node.js
• nodemailer
• nvidia-cosmos
• openai-api
• qrcode
• react
• sharp
• smtp
• tailwindcss
• typescript
• upi
• vllm