PocketOracle

Inspiration

We live in an age of data abundance but information poverty. Even with a dozen financial apps on our phones, finding a straight answer to a simple question is surprisingly difficult. Most tools are great at hoarding data, listing every latte and lunch, but they fail at contextual organization and proper retrieval.

The problem isn't a lack of records; it's that the information is fragmented. Your digital transactions are in one app, your cash spending is a mystery in your head, and your physical receipts are crumpled in a drawer. Existing solutions provide "analytics" that are buried under layers of menus, making it impossible to retrieve the one piece of context you actually need when you're standing at a checkout counter. We built Pocket Oracle because we believe that financial clarity comes from a perfectly organized, unified state that is instantly retrievable through natural conversation.

What it does

Pocket Oracle is an AI-powered personal accountant that reconstructs a user’s Unified Financial State, bridging the gap between digital banking and physical cash.

Unified Financial State Engine: It combines live bank transactions with a tracked "Cash Pool." It recognizes that an ATM withdrawal is a transfer to a physical wallet, not an immediate expense.
Multimodal Receipt Understanding: Users snap photos of receipts, and the system performs high-fidelity extraction of merchant and amount data to accurately deplete the cash pool in real-time.
The Affordability Advisor: Instead of generic advice, this specialist agent calculates a real-time "Affordability Score" and simulates the impact of a purchase on your future liabilities.
Active Reconciliation: It detects when your modeled balance diverges from your real bank balance and prompts you to reconcile missing expenses, ensuring the data is always the "single source of truth."

How we built it

We prioritized a modular, agent-based architecture focused on data integrity and organization.

Financial Data Layer: We integrated the Plaid Sandbox to fetch digital transactions. We built a custom state engine to handle the internal movement of funds: $$S_{total} = B_{plaid} + C_{pool} - E_{untracked}$$
Multimodal Extraction Pipeline: We utilized Gemini Vision to bypass traditional, rigid OCR. This allows the system to parse complex, unstructured receipt images and identify intent, merchant, and totals with zero-shot accuracy.
Multi-Agent System: We created specialized AI agents (Spending Analyst, Affordability Advisor, Scenario Simulator) that share the same structured financial summary to ensure consistent reasoning.
Frontend: We designed a Decision-First UI. Instead of a homepage full of graphs, the "Decision Center" hero section allows users to simulate potential purchases for an immediate risk analysis.

Challenges we ran into

The Synchronization Mismatch: The hardest challenge was handling the temporal delay between an ATM withdrawal appearing in Plaid and the cash entering the user's wallet. We built a state machine to hold these transactions in "Pending Allocation" to prevent double-counting.
Hallucination vs. Math: Large Language Models are great at reasoning but can be poor at arithmetic. We separated the logic: we use deterministic Python functions for the heavy mathematical lifting and pass those hard numbers to the agents to generate the empathetic explanation.
Data Fragmentation: Ensuring that data from three different sources (API, Image-based extraction, and User Input) remained organized and retrievable without conflict was our major architectural hurdle.

Accomplishments that we're proud of

The "Cash Pool" Logic: We successfully built a system that tracks cash as a transferable asset rather than an immediate loss, solving a major pain point in personal finance.
High-Speed Retrieval: We are proud of how quickly our agents can query the "Unified State" to give a definitive, evidence-based answer on purchase impact.
The Agent Persona: We managed to tune the Financial Coach agent so it feels proactive and insightful rather than robotic, identifying "subscription clusters" and spending trends automatically.

What we learned

Organization is Strategy: We learned that simply "having the data" isn't enough. The way information is structured for retrieval determines whether an AI can actually provide guidance or just "hallucinate" over a spreadsheet.
User Psychology: We learned that users find more value in forward-looking simulations (what will happen) than in backward-looking analytics (what did happen).
The Complexity of Cash: Integrating physical cash into a digital model is technically difficult but necessary for a complete, honest financial picture.

What's next for Pocket Oracle

Voice Interface: Implementing a hands-free "Oracle" mode for instant "Can I buy this?" queries while shopping.
Predictive Organizing: Using AI to predict upcoming cash needs based on historical patterns, suggesting when you should visit an ATM.
Proactive Subscription Negotiation: Empowering the agents to not just identify subscriptions, but draft cancellation emails or negotiation scripts for the user.