AI BookKeeper

Inspiration

I hate doing my books. Not because numbers scare me - I actually like spreadsheets. I hate it because the work is mindless. Every month: open the bank export, paste into a sheet, drag categories down, hunt for the receipt that doesn't match, wonder if that $3,200 wire was the vendor or a duplicate, give up and mark it "miscellaneous." Repeat.

At some point I started thinking - this isn't financial problem. It's a pattern-matching problem. My inbox already has the receipts. My Drive already has the expense sheets. The bank already knows what came in and that went out. All the information exists. It's just sitting in three different places, waiting for someone (me, apparently, at 11pm) to manually stitch it together.

I'd been watching the accounting profession quietly hollowing out - 300,000 accounts gone since 2020, most of the rest approaching retirement - while small business owners got no real alternative beyond expensive software that still requires someone to run it. The gap between what business need and what they can actually afford kept widening. The felt like the problem worth solving, and AI reasoning models had finally gotten good enough to actually solve it. So I build the thing that should already exist.

What it does

AI BookKeeper connects to your Gmail, Google Drive, and bank feed, then runs continuously in the background doing the work you'd normally do yourself on a Sunday night. It reads your inbox for receipts and invoices, pulls in your expense spreadsheets from Drive, and merges everything into a single live transaction view - categorized, reconciled, and flagged for anything that looks off. Duplicate payments, unusual vendor charges, wire transfers with no paper trail - it catches them before you do, and it tells you why they're suspicious, not just that they are.

Ask it anything in plain English and it answers from your actual data. "why did costs spike in March?" gets you the real answer: a new engineering hire plus an annual software license renewal, totaling $9,000. Not a generic response - the specific numbers.

Budget rules let you set spending caps per category and see exactly how many days until you breach them at the current burn rate. Drag a slider to model a pessimistic month and watch the breach dates move in real time. The system warns you before the problem happens, not after.

And if you want to teach it your patterns = "always categorize Uber as travel", "split GlobalTravel invoices 70/30 between travel and entertainment" - you just type it. The agent learns your business instead of the other way around.

How we built it

We didn't plan seven features from the start. We built one thing, hit a wall, built the thing needed to get past it, and kept going.

It started with a clean transaction list - debit/credit colours, category icons, a filter by expense type. Nothing clever, just making the data readable without an accountant to interpret it. The hard part at this stage was resisting the urge to add things before the foundation actually worked.

Anomaly detection came next, because once you can see the transactions the obvious question is: what's wrong here? Three real-world patterns shaped the whole severity system - a vendor charged twice in the same week, a travel expenses three times the monthly average, a wire with no invoice. Those cases ended up driving almost every design decision that followed.

Then the data sources. Connecting Gmail meant parsing invoice emails and payment confirmations - not just subject lines, but pulling amounts, vendors and dates out of the body. Connecting Drive meant ingesting spreadsheet rows and reconciling them against what the bank said. That reconciliation immediately surfaced something interesting: four SaaS subscriptions totaling $620/month were showing up in Gmail receipts but nowhere in the expense sheets. Nobody knew they were still running.

The six-month trend chart came after that, because monthly snapshots weren't enough. It showed something the individual months were hiding: revenue up 33%, but costs growing faster. A margin compression story that wouldn't appear in a normal P&L until it was already a problem.

Then we wired in Gemini. Instead of a hardcoded system prompt, we built a function that assembles the full financial context fresh on every question - active sources, current totals, flagged items, offline warnings. That's what made it feel like a reporting tool and more like something that actually thinks.

Challenges we ran into

Getting the AI to reason from live data, not stale snapshots. Early on the answers were good but frozen — they reflected whatever the data looked like at build time. The fix was treating the system prompt as a function that runs fresh on every query, reading active sources and computing current totals each time. If Gmail goes offline mid-session, the next answer acknowledges the gap instead of confidently drawing from data it no longer has. Small architectural change, massive difference in whether you can actually trust the output. Deciding which source to trust when they disagree. When Gmail shows $8,640 in software costs and a Drive spreadsheet implies a different number, you don't have an answer — you have a conflict. The instinct was to resolve it quietly and move on. The right call was to surface it and let the user decide. That $3,200 discrepancy turned out to be the most valuable thing the whole system found. Hiding it automatically would have been worse than finding nothing. Parsing natural language rules without a library. Getting "split GlobalTravel invoices 50% travel 50% entertainment" to produce two category allocations, a vendor filter, and correct percentages — without any NLP tooling — meant building a keyword extractor from scratch. Vendor names, dollar thresholds, category words, time scopes, action verbs — assembled into structured logic on the fly. The edge cases took longer than the happy path, as they always do. Keeping everything reactive without the state becoming a mess. The days-to-breach number, the projection chart, the warning feed, and the overview metrics all need to update the instant a slider moves. Five categories, four sliders, three chart views — reactive state gets tangled fast. The solution was one function that recalculates everything from scratch on every change rather than trying to surgically update individual pieces. Slightly less efficient, dramatically easier to reason about. Stopping the model from filling in gaps with plausible-sounding nonsense. Language models want to be helpful. Given a question and incomplete data, they'll complete the picture with something reasonable. Fine for a chatbot, dangerous for a tool people use to make financial decisions. Hard rules in the system prompt — only cite numbers present in the context, always name the source — fixed most of it. Pre-structuring the anomaly descriptions so the agent is describing known flags rather than generating new ones fixed the rest.

Accomplishments that we're proud of

Getting the cross-source reconciliation to actually work. Merging a live bank feed, Gmail receipt parsing, and Drive spreadsheets into one coherent transaction view — with conflicts surfaced rather than silently resolved — is genuinely harder than it sounds, and it's the part that makes the whole thing trustworthy rather than just impressive-looking. The natural language rules engine. Typing "always categorize Uber as travel" and watching the transaction list update immediately, each match tagged with the rule number that applied — that interaction felt right in a way that took several attempts to get to. Most tools make you learn their language. This one learns yours. The predictive breach warnings. "You'll hit your software cap in 6 days at the current burn rate" is a different category of useful than "you've used 86% of your budget." One is a number. The other is a decision prompt. Getting that distinction right, and backing it with a scenario modeller that updates in real time, is the thing we're most pleased with from a product standpoint. Building it without a team. Every layer of this — the data pipeline, the AI context engine, the rules system, the UI — was put together by one person who doesn't do cold calls and wasn't planning to pitch anyone. That constraint shaped the product in ways that made it better.

What we learned

The value isn't in the features. It's in what happens between them. A transaction list is just a spreadsheet. Add anomaly detection and it becomes a watchdog. Add Gmail parsing and the watchdog has more to watch. Add trend analysis and you can see where things are heading. Add AI reasoning and you can ask why. Add budget rules and you get warned before the why turns expensive. Add natural language rules and the system starts learning your business instead of the other way around. None of those individually is that interesting. Stacked together they produce something that didn't really exist before — a bookkeeper that doesn't sleep, doesn't miss patterns, and gets a little smarter every time you tell it something new. We also learned that "making the AI trustworthy" is a harder problem than "making the AI capable." Getting Claude to reason well over financial data took a day. Getting it to acknowledge uncertainty, name its sources, and not invent numbers when the context is incomplete took considerably longer. That's the work that actually matters for a tool people are going to rely on.

What's next for AI BookKeeper

QuickBooks and Xero sync. The biggest objection from established businesses is "I already have a system." The answer is becoming an AI layer on top of whatever they're already using, not a replacement. Two-way sync — pull their existing books in, push categorizations back out — removes that objection entirely. Invoice generation. Right now the agent reads invoices. The next step is creating and sending them from inside the same tool. That closes the loop between tracking income and actually generating it, and it makes the app a daily habit rather than a monthly review. WhatsApp and Telegram alerts. When a budget cap is about to breach or an anomaly gets flagged, the right place for that notification isn't an email — it's a message. A quick async ping that requires no response, no login, no context-switching. That's the alert system that actually gets seen. Multi-company support. One user, multiple business accounts, one dashboard. This is the feature that turns a solo tool into something an accountant can use across their whole client list — and the one that makes the white-label model viable. Receipt camera scanning. Phone camera, photo of a paper receipt, categorized transaction in under five seconds. It's the last manual data entry task that the current version doesn't solve, and for businesses that still deal in cash it's the most important one. The goal throughout has been the same as it was at the start: build the bookkeeping tool that actually handles it, so you don't have to be doing your books at 11pm. We're not done yet.

Built With

• axios
• date-fns
• hook
• jspdf
• lucide-react
• miaoda-auth-react
• miaoda-sc-plugin
• motion
• pnpm
• qrcode
• radix
• react
• react-day-picker
• recharts
• router
• shadcn/ui
• streamdown
• supabase
• tailwind
• typescript
• vaul
• vite
• zod