PennyPal

Inspiration

Most financial apps treat every conversation like it's your first. You open a budgeting tool, and it has no idea you've been saving for something for three months. It doesn't remember that you always overspend on weekends, or that you told it last week you were stressed about rent. You have to re-explain yourself every single time.

I wanted to build something that actually listens and remembers. Not just within one conversation, but across days, weeks, and sessions. The kind of financial coach that gets sharper the longer you use it, because it knows your story, not just your last message. That gap between "stateless chatbot" and "coach that actually knows you" is what PennyPal was built to close.

What it does

PennyPal is a persistent memory financial coaching agent. Users chat with it about their money, their goals, their habits, and their struggles. PennyPal remembers what matters across completely separate sessions and uses those memories to give increasingly personal, accurate coaching over time.

It remembers five categories of information: financial goals, spending habits and behaviors, emotional attitudes toward money, hard constraints like salary dates and monthly bills, and action plan commitments the user makes to themselves.

It knows when to forget. Passing feelings and habits slowly decay in importance if not mentioned again. Goals and hard constraints stay permanent until the user explicitly changes them. When a user changes their mind, the old memory is archived and linked to the new one rather than deleted, preserving the full story of how their thinking evolved.

It also knows when to speak up. If PennyPal detects a pattern across multiple memories, like anxiety about money mentioned three separate times, it surfaces that pattern proactively rather than waiting to be asked.

How we built it

PennyPal runs on a Flask backend deployed on Alibaba Cloud ECS in the Singapore region, with a plain HTML and JavaScript frontend that connects to the backend over HTTP.

The memory system uses Supabase as a Postgres database with two core tables. Episodic memories store specific timestamped events with category labels, importance scores, decay scores, and status fields that track whether a memory is active, archived, superseded, or deleted. Semantic facts store durable distilled truths about the user, like a confirmed savings goal or a recurring income pattern, with confidence scores and supersession links when facts change.

The most deliberate engineering decision was the two model architecture. Every conversation turn runs through two separate Qwen API calls. The first call uses qwen-max to generate the coaching response, with relevant memories retrieved from Supabase and injected into the prompt context. The second call uses qwen-plus to extract what is worth remembering from the same exchange, returning structured JSON with category, importance score, and whether anything contradicts an existing memory. qwen-max handles quality where it matters. qwen-plus handles speed and cost efficiency in the background where the task is simpler.

Memory retrieval is capped at the top five to eight episodic memories per turn, ranked by a combined importance and decay score, plus all active semantic facts. This cap is the practical answer to recalling critical memories within a limited context window without flooding it with everything ever stored.

The frontend includes a live memory inspector panel showing exactly what PennyPal currently remembers, what it just retrieved for the current message, and what it just stored or archived after the response. There is also a memory on and off toggle that makes the difference between stateful and stateless coaching visible in real time.

Challenges we ran into

The hardest problem was not building memory storage. That part is straightforward. The hard part was building memory that actually changes behavior in a visible and meaningful way, not just memory that exists silently in a database.

Getting the memory extraction prompt right took significant iteration. qwen-plus needed very specific instructions to return clean structured JSON consistently, correctly identify which category a piece of information belonged to, and flag contradictions against existing memories rather than just appending new ones alongside old ones.

The decay system required careful thinking about what should fade and what should not. A single mention of weekend spending anxiety should lose relevance over time if never mentioned again. A confirmed savings goal should not fade just because three sessions passed without discussing it. Getting those rules right at the category level, rather than applying a single blanket decay rate, was a non-trivial design problem.

Deploying to Alibaba Cloud ECS and configuring the server to accept outside traffic correctly, including opening the right ports in the security group and keeping the Flask process running persistently after closing the terminal, was a practical infrastructure challenge that had to be solved from scratch.

Accomplishments that we're proud of

The two model architecture is the accomplishment I am most proud of. Most hackathon submissions use a single model call per turn. Using qwen-max for response quality and qwen-plus for background extraction is a real engineering optimization decision, not just a feature, and it directly reflects how a production system would be designed under cost and latency constraints.

The supersession linking system, where changing a goal archives the old memory and connects it to the new one with a reference link, means PennyPal can tell a user "you upgraded your goal from saving 50,000 naira to 150,000 naira" rather than just silently having a new number with no story attached. That small detail makes the difference between memory that exists and memory that actually coaches. The memory inspector panel makes the invisible visible. Judges and users do not have to take our word for it that memory is working. They can watch it happen in real time.

What we learned

The biggest lesson was that memory is not a storage problem. It is a retrieval and forgetting problem. Anyone can save text to a database. The hard question is: which five things out of forty stored memories actually matter for this specific message, right now? Building a system that answers that question deliberately, through importance scoring, decay, category weighting, and relevance retrieval, rather than just dumping everything into context, is where the real work lives. The second lesson was about demo design. A feature that exists in code but cannot be shown clearly in thirty seconds of video does not exist for a judge. The memory on and off toggle, the live inspector panel, and the session contrast moment were not afterthoughts. They were designed specifically so that the mechanism is visible, not just the output.

Built With

• alibaba-cloud-ecs
• css
• flask
• html
• postgresql
• python
• qwen-cloud-(qwen-max-and-qwen-plus-via-dashscope-api)
• supabase