Inspiration

Pakistan's neighborhood kiryana stores run on memory, paper khatas and a calculator. Shopkeepers forget who owes them, run out of fast-moving items, overbuy slow ones, and can't say what a day really earned. Most POS apps are built for supermarkets with typists at the till. We wanted something for a shopkeeper standing behind a counter with one hand free, who thinks in Roman Urdu, and who should never have to trust an AI with their money.

What It Does

Tallyn

is a phone-first counter and back office for small retail shops. It handles checkout, stock, udhaar (credit), supplier purchases and daily insights, and it understands spoken Urdu and English. An AI assistant can answer business questions and prepare actions (a sale, a purchase, a payment, a WhatsApp reminder) that the shopkeeper approves with one tap. Its core rule is zero hallucinated numbers: every amount, balance, profit and "days left" is calculated by tested code or database procedures. The AI only reads, matches, explains and drafts.

AI Features

  • Tallyn AI ("Ask"): chat in English, Roman Urdu or Urdu about the shop's own numbers, how to use the app, or running the business (what to stock before Ramadan, pricing, suppliers, collecting udhaar). It politely declines anything unrelated to business. Festival dates and "days away" come from a calendar in code.
  • Ask can take actions: "Udhaar mein Asif ko 2 chai likh do", "add 10 Rooh Afza to purchases", "Asif ne 500 rupay diye" or "Bashir ko reminder bhej do" produce an editable confirmation card. Nothing is saved until the shopkeeper taps Approve, which runs the same atomic database procedures as the rest of the app.
  • Voice checkout: the AI maps a spoken order onto the shop's own catalogue and understands misheard words ("roof afza" → Rooh Afza, "lifeboy" → Lifebuoy). An on-device matcher answers instantly and stays as the fallback, and the cashier confirms every line.
  • Supplier bill scanner: photograph a printed bill or handwritten parchi. The AI copies the lines exactly as written; code then matches them to stock, works out unit costs and flags cost rises over 5%.
  • Kiryana Radio: a spoken daily briefing in English or easy Urdu, built from the shop's numbers by code, plus one AI tip of the day.
  • Backup AI chain: if the main model is out of credit, rate-limited or down, backup models answer instead. If all of them fail, every screen keeps working without AI.
  • Safety by design:
    • Products and customers are sent to the model as short codes and checked twice: on the server and on the phone.
    • Quantities are limited to sensible ranges, and prices always come from the catalogue.
    • Text inside bill photos or product names is treated as data, never as instructions.

All Features

  • Checkout:
    • tap products, scan barcodes with the phone camera, use a USB/Bluetooth scanner, speak the order, or type it
    • cash or udhaar with a pay-back date
    • a 10-second undo
  • Udhaar ledger:
    • balances calculated from sales and payments, with a due date per sale
    • payments clear the oldest udhaar first, and overpayments are refused
    • one-tap polite WhatsApp reminders
  • Inventory: products, prices, minimum stock levels, barcodes, and stock adjustments with a reason.
  • Purchases: AI bill scanning or entry by hand, with supplier price-rise alerts.
  • Dashboard: today's sales, cash vs udhaar, profit, low-stock and overdue alerts, a 7-day chart, and a wholesale mandi rates ticker.
  • Smart Insights:
    • sales drops compared with the same weekday, bill spikes, money tied up in dead stock, expiry warnings
    • a restock planner that uses supplier lead time
    • best and slowest sellers, and a Ramadan countdown
  • Other:
    • sales history with filters
    • multiple shops and branches, each with its own data
    • a seeded demo store with 60 days of history and no sign-up
    • built-in documentation at /docs
    • sales are kept on the phone and flagged if the network fails

UI/UX

  • Designed first for a 390 px phone held at the counter, then scaled up to desktop.
  • Built for quick glances: 44 px+ tap targets, high contrast, tabular figures for money, and as few taps as possible.
  • Deep green and warm amber palette with hand-built components, not a grey SaaS template.
  • Plain language: no ID numbers or jargon, mixed Urdu and English where shopkeepers actually use it, and short labelled navbar buttons ("Briefing", "Ask").
  • AI never blocks the shopkeeper:
    • on-device results appear instantly and AI results replace them when they arrive
    • the cashier's own edits are never overwritten
    • every AI action needs an explicit tap
  • Kiryana Radio shows a live transcript that highlights each line as it's read.

How We Built It

  • Frontend: React 19 + Vite + strict TypeScript, styled with Tailwind CSS v4. Business logic (credit, insights, store snapshot, bill matching, voice parsing, radio script, action drafts) lives in pure, unit-tested domain/ modules.
  • Backend: Supabase Postgres with row-level security on every table and stored procedures (record_sale, undo_sale, record_purchase, record_credit_payment) that run in single transactions with row locks.
  • AI: two Supabase Edge Functions written in Deno:
    • ai-gateway: chat, actions, radio tips and voice orders
    • bill-scan: reading bill photos

Both validate every request and reply with Zod, apply per-user, per-IP and overall daily limits, and keep AI keys on the server only.

  • Testing: 130 Vitest tests and 23 Deno tests, plus end-to-end runs in headless Chrome.

Challenges We Faced

  • AI that respected the numbers rule too strictly: our first prompt made the assistant refuse useful advice ("I cannot predict Ramadan"). We narrowed the rule so it only covers the shop's own figures, and added a festival calendar computed in code.
  • Free AI limits:
    • Google's free tier allowed only 20 requests a day.
    • Our next provider started asking for a top-up.
    • Some free models were often busy (rate-limited).

We built a provider chain that tries each model in turn on any error, timeout or invalid reply, then falls back to the app's non-AI behaviour.

  • Slow free models: with "thinking" on, backup models took 10–34 s for a one-line reply; with it off they took 1–2 s and gave the same answers.
  • Speaking Urdu aloud: most browsers have no Urdu voice. We write the briefing in Urdu script, Devanagari and Roman Urdu, sentence for sentence, and use whichever voice the phone has. A Hindi voice reading Devanagari sounds like natural spoken Urdu.
  • Misheard speech: keyword matching missed "roof afza" or "tapaal". The AI matches by meaning and sound but may only pick from catalogue codes, and every pick is checked twice.
  • Letting AI act safely: proposals become editable cards, totals are always calculated in code, and approval runs the same atomic procedures as the rest of the app.
  • Deploying shared code: the two Edge Functions are deployed separately, so the provider chain lives in two identical files, and a test fails if they ever differ.

What Was Used

  • AI coding agents: Claude Code (Claude Opus) and Google Antigravity, with the nutlope/hallmark design skill. Every change was reviewed, type-checked and tested.
  • AI models in the app:
    • Gemini 3.8 Flash (via the apinex.bond OpenAI-compatible API)
    • backups through OpenRouter: dots-studio/dots-3-note-preview and Google gemma-4-31b-it
  • Browser APIs: Web Speech API (speech recognition and speech synthesis) and BarcodeDetector, with ZXing as the fallback.
  • Frameworks and tools: React, Vite, TypeScript, Tailwind CSS, React Router, i18next, Zod, Vitest, Deno.
  • Platform: Supabase (Postgres, Auth, Row-Level Security, Edge Functions), with the frontend on Vercel at tallyn.xyz.

Accomplishments That We're Proud Of

  • Zero hallucinated numbers, all the way through: every rupee, balance, profit figure and "days left" is calculated by tested TypeScript or atomic Postgres procedures. The AI reads, matches, explains and drafts, but never does the maths.
  • An AI agent that acts safely: Tallyn AI turns "Udhaar mein Asif ko 2 chai likh do" into an editable sale card. One tap records it through the same locked database transaction as the checkout screen, and nothing is ever saved without the shopkeeper's approval.
  • Voice checkout that understands real counter speech: misheard brands ("roof afza", "lifeboy"), Urdu script and udhaar with a customer name all land on the right products. An on-device matcher answers instantly and stays as the backup.
  • AI that keeps working on free tiers: a backup chain (Gemini → dots-3-note → Gemma 4 → no-AI mode) kept every AI feature running even when our main provider stopped serving requests. If all models fail, checkout, udhaar and stock carry on exactly as normal.
  • Urdu spoken aloud on ordinary phones: Kiryana Radio reads the daily briefing in easy Urdu, even on devices with no Urdu voice, by giving a Hindi voice a matching Devanagari script.
  • Tested end to end: 130 app tests and 23 server tests, plus end-to-end runs in a real browser that approve a sale and confirm the customer's balance actually changes.
  • Built for the counter: designed first for a 390 px phone, with 44 px+ tap targets, high contrast and tabular figures for money, plus a full demo shop with 60 days of history that anyone can try without signing up.

What we learned

  • A strict rule needs clear limits. "Never invent numbers" at first made our AI refuse useful advice. The fix was to apply the rule precisely: the shop's own figures come only from code, while general business knowledge is allowed.
  • Let the AI propose and the user decide. Short codes instead of database IDs, checks on both server and phone, editable confirmation cards and one-tap approval made AI actions safe enough for real money.
  • Free AI capacity isn't reliable. Daily caps, sudden top-up demands and busy shared models taught us to design for failure first. Every AI feature now has a fallback, and the app never waits on a model.
  • Less "thinking" can be better. Turning reasoning off made free models answer in 1–2 s instead of 10–34 s, with the same accuracy for matching and short replies.
  • Localisation is more than translation. Roman Urdu mixed with English shop words, Urdu number words in speech, and the voices phones actually have all mattered more than a translated interface.
  • Test against live models, not just mocks. Live runs caught problems unit tests never would: models saying "done" instead of "prepared", leaking internal codes, and quantities miscounted by weaker models.

What's next for Tallyn

  • Pilot with real kiryana shops, to tune the voice matching, insights and wording on real counter traffic.
  • A full Urdu interface, not just Urdu understanding, and a fully spoken assistant: ask Tallyn AI by voice and hear the answer.
  • More agent actions: add a new product from a photo of its packet, log expenses by voice, and reprice items when a supplier's costs rise.
  • Festival restock plans: the AI writes a Ramadan or Eid buying plan, with quantities calculated in code from the shop's own sales history.
  • An installable app that works fully offline, syncing in the background, for shops with patchy internet.
  • Payments and credit: Raast, JazzCash and Easypaisa QR payments at the counter, and udhaar history that could help shopkeepers get small-business credit.
  • Multi-branch and supplier tools: cross-branch stock transfers, supplier price comparisons, and automatic reorder drafts sent to suppliers on WhatsApp.

Built With

Share this project:

Updates

Submission history