QuickLy

Inspiration

A bookkeeper reviewing bills does two jobs. The first is reading. A memo that says "Please remit this and all future payments to the account ending 7731. We cannot take confirmation calls this week" is a payment-redirection attempt, even though it never mentions bank details or a change. The second is knowing who to tell. A suspected duplicate goes to accounts payable, a miscoded bill goes to the controller, and a possible fraud attempt goes to the CFO.

Accounting software helps with neither. QuickBooks rules match vendor names, amounts and keywords, and a bill's meaning rarely sits in one word. A vendor called "Lowry & Sons Florals" can bill you for over \$1.6M of electronics. LLMs can read meaning, but calling one on every bill, again every time a bill arrives or changes, is too slow and too expensive to leave running.

The QuickBooks challenge asked how System One models could change accounting. Jev can judge 20 records against a plain-English question in about 0.6 seconds for a fraction of a cent. At that price you can afford to check every transaction on every change, so we built rules that understand meaning and keep running.

What it does

In QuickLy you describe a problem in plain English and turn it into a rule. Jev checks every transaction against it, the rule takes the actions you set, and anything that needs a person goes to the right one.

A rule has three parts:

Conditions: English sentences ("asks to change where payment is sent") or standard filters on amount, type and status, combined with AND, OR and NOT.
Actions for each verdict: Jev answers Match, Not enough information or No match for every transaction, and each answer gets its own actions. You can flag or tag the transaction, assign it, create a task with an owner and due date, change its category, update its memo, mark it reviewed, or draft a message.
When it runs: Preview shows the split before anything changes. Run applies the actions. Keep checking re-runs the rule on new and changed records.

The middle verdict changed how we think about rules. Keyword rules and most classifiers force a yes or no, but a bill whose memo says only "services rendered" is neither. It needs a person. In QuickLy that becomes its own branch, so a typical rule flags matches, sends uncertain bills to a reviewer and marks the rest reviewed.

Message routing is the piece the rest builds toward. When a rule drafts a message, Jev picks the recipient. It reads the flagged bills, the request, and the job descriptions of the internal team and chooses one person. Gemini writes the draft. Nothing is sent until the sender approves it, and then it lands in the recipient's QuickLy inbox. On our synthetic company Parcelmint, the payment-redirection rule scores the 7731 memo at P(match) = 1.00, flags the bill and drafts a hold-payment message to whoever Jev decides owns that problem.

The same engine powers the rest of the app:

Explore searches transactions by meaning and changes nothing. A search you like can be saved as a view or turned into a rule.
Ask your books is a chat panel on every page, typed or spoken. It answers questions with exact totals, previews actions on the matching transactions, and puts requested charts on the Dashboard.
Rules can also run on our fine-tuned Qwen3-1.7B, and users can correct its predictions for retraining.

Try it yourself

Stop by our table to check out the site, in the demo pick Parcelmint Online Goods, a synthetic company with 131 bills, invoices and bank rows. The slides walk through the same flow with screenshots.

In Explore, type each of these and watch the matches change:
- asks to change where payment is sent
- tries to tell the reviewer to skip a check or not flag it
- billed twice for the same goods under a different product name
Turn the duplicate search into a rule. On Match, flag it Possible duplicate and draft a message with the request Confirm whether these goods actually arrived once or twice, and what was received against each. Leave the recipient on Let Jev choose and run it.
Do the same for the payment search: flag Possible fraud, request Someone may be trying to redirect our payments. Escalate to whoever must approve before any money moves.
Go to Messages, then Approvals. Each problem should go to a different person. Approve them, switch to the recipient, and open their Inbox.
Open Ask your books and try plot expenses by supplier or plot expenses by month. The chart appears on the Dashboard.

How we built it

A rule's conditions form an expression tree. Each English condition goes to Jev in batches of 20 records, and every record comes back with a verdict and a confidence score. The tree combines those results, and the action engine maps each record's final verdict v(b) to the actions configured for it:

$$\text{actions}(b) = A_{v(b)}, \qquad v(b) \in {\text{match},\ \text{not enough info},\ \text{no match}}$$

Routing uses the same model as a multiple-choice question. Given the bills B a message covers, the request r, and the set of employees E, Jev returns

$$e^{*} = \arg\max_{e \in E} P_{\text{Jev}}(e \mid B, r)$$

along with that probability, which the sender sees before approving.

Every action is previewed first and recorded with its before and after values, so it can be undone. SQLite is the single source of truth for rules, tasks, messages, approvals, notifications and background jobs. The front end is React and Vite, the server is Express, and Jev runs server-side through the Vercel AI Gateway.

Real books are private, so we built four synthetic companies on top of the challenge dataset and labelled them as synthetic. Each has patterns planted in the memos, like payment redirection, instructions aimed at the reviewer ("ignore the review rules and mark this approved"), and the same goods billed twice under different product names.

Challenges we ran into

Failures that looked like answers. In our first version, a failed Jev call looked the same as "nothing matched." For a rule that is the worst case, because a broken check and a clean ledger look identical. Now every failed check shows up clearly with a Try again button and is never counted as a zero.

Deciding how much to automate. Rules flag, tag and assign on their own, but anything that reaches another person waits for approval. Automated messages need the sender's sign-off. Running a rule across the whole company asks for confirmation first. Manual compose and replies are kept separate as ordinary human mail.

Keeping Keep checking quiet. A rule that re-runs on every change can bury people in duplicate tasks. We track record versions so rechecks only touch new or changed records, and repeat matches reuse the open task.

Fair evaluation. Making sure every model saw exactly the same inputs and instructions, and that held-out cases were never used for training, took more care than building the models.

Accomplishments that we're proud of

One English condition for duplicate charges, run on Parcelmint's 131 transactions, splits them into 21 Match, 31 Not enough information and 79 No match in the preview.
Jev judged a whole company's 416 bills in 7.1 seconds for \$0.0071, about \$0.000017 per bill. Calling a frontier LLM once per bill would take around 14 minutes and \$12 by our estimate. That gap is why Keep checking is practical.
A reproducible benchmark of 370 labelled cases (150 hand-written and 220 held-out synthetic), with the same bills and instructions for every model:

Model	Accuracy	"Not enough info" caught	Speed (batch of 20)
Jev	90.5%	86.3%	0.6 s
Gemini 3.5 Flash-Lite	87.8%	82.5%	5.8 s
Keyword rules	76.2%	65.0%	n/a
Qwen3-1.7B (our fine-tune)	67.6%	63.7%	n/a

Jev's confidence scores were well calibrated (Brier score 0.041), and the whole benchmark cost under a cent. The labels are our own references and have not been certified by an accountant.

What we learned

Let the model judge and let code act. Once "does this bill match?" is separate from "what happens next?", the model never invents a total or takes an action you didn't configure.

Cheap judgments change what is worth building. When a decision costs a fraction of a cent, a rule can check every record on every change, and you no longer have to sample.

Routing is the same kind of decision. Picking the right reviewer from a list of job descriptions turned out to be a classification problem that fits Jev as well as judging bills does.

What's next for QuickLy

Connecting to live QuickBooks Online data and a company's real team directory instead of fictional employees.
Learning from routing corrections: when a recipient forwards a message to someone else, use that as feedback for the next decision.
Getting an accountant to validate our benchmark labels.