Inspiration
- I wanted one place to track spending, investments, EMIs, loans, debts, recurring payments, limits, and who owes whom.
- I wanted to search naturally, like “coffee,” and instantly find every related transaction.
- All of my transactions should be neatly categorized, food, books discretionary expenses etc.
- I wanted deterministic heatmaps, graphs, safe-to-spend numbers, and useful insights. Calendar marking due date of loans, upcoming emi payments etc.
- I wanted the system to remember personal rules, like splitting every Goa trip expense with Alice and Bob or rent with Priya.
- Most importantly, I wanted to do all of this voice-first, without needing to maintain another spreadsheet or constantly type into an app. In any of my languages that too, Telugu, Hindi etc.
What it does
- TLDR: everything above
- Knot records and categorises expenses, income, investments, loans, EMIs, recurring payments, limits, debts, and receivables through natural language or voice.
- It gives deterministic analytics including spending heatmaps, essential vs discretionary spending, safe-to-spend, portfolio values, amortisation, and monthly-limit pacing.
- It supports semantic transaction search, so queries like “coffee” can retrieve related transactions even when the exact wording differs.
- Knot remembers facts and rules about the user, applies them later, and shows exactly which memories it used.
- It keeps an append-only double-entry ledger, so corrections create reversing entries instead of silently rewriting financial history.
How we built it
- Knot uses a small hand-written agent loop where the model understands intent and chooses among 38 tools, while SQL performs all accountin.
- CockroachDB stores an append-only double-entry ledger, with balances always derived from ledger entries.
- We use CoALA: working memory for conversation context, episodic memory for events, semantic memory for user facts, and procedural memory for rules such as expense splits.
- Semantic and procedural memories use vector retrieval, while memory traces make every retrieved fact or rule inspectable by the user.
- The backend runs on AWS App Runner, container images live in ECR, logs go to CloudWatch, and text and voice both use the exact same tool registry.
Challenges we ran into
- Prevent the LLM from becoming the accountant.
- One early memory demo appeared to work even though no memory had been retrieved; the model had copied names from a prompt example, which led us to make memory traces part of both the product and debugging workflow.
- Procedural rules were difficult to retrieve reliably from differently worded future requests, so we embed both the rule trigger and instruction and measured retrieval thresholds rather than guessing them.
- Correcting memories and financial mistakes without destroying history required superseded memories and reversing ledger entries instead of deletion or mutation.
- We also had to fix semantic mistakes in analytics, such as initially treating investments as spending and generating technically correct but useless observations about established habits.
Accomplishments that we're proud of
- Accessibility: Every feature is fully keyboard navigable with a simple Tab to Enter flow, usable hands-free through voice, and built with screen-reader-friendly labels and structure.
- Can be taught a financial rule once, recall it later from different wording, apply it automatically, and show the exact memories responsible. Provenance first.
- The ledger remains balanced even under concurrent writes, with serializable transactions.
- There is no mutable balance field. Append only, tool calls to void entries,
- Github style heatmap of spending.
- Visualisations encode financial meaning, are colour coded: investments do not count as spending, no-spend days remain neutral, and heatmap intensity reflects a user's normal spending distribution rather than one extreme purchase.
- Voice invokes the same tools, memory system, etc.
What we learned
- Practical experience of using Isolation levels in DB.
- Keep backend and DB in same AWS region to reduce round trip time.
- Cockroach DB, its consistency model, multi-active availability and so on. CRDB also has great docs and a great technical evangelist.
- CoALA memory model for designing good agentic memory.
- Importance of provenance for financial data.
- Controlling what information enters the model's context.
- Many many design decisions, eg. split 100 among alice, bob, and I. 100/3 + 100/3 + 100/3 would be 99.99. Doubly entry ledger would not sum to zero, instead receivable by alice and bob stay 33.33, while money paid by me, is 100 - 66.66, so 33.34
- The architecture that worked best was simple: the model understands, memory supplies context, tools define actions, SQL calculates, and the ledger remains the source of truth.
What's next for Knot
- Connect banks and brokerages so transactions, investments, loans, and recurring payments can be reconciled automatically against the ledger.
- Expand shared finance workflows for roommates, couples, households, and trips.
Built With
- ann
- aws-app-runner
- aws-cloud-watch
- aws-ecr
- coala-agentic-memory
- cockroachdb
- embeddings
- nextjs
- openai-real-time
- python
- sse
- websockets
Log in or sign up for Devpost to join the conversation.