WhatsMyFin

WhatsMyFin is an AI powered Modern Personal Finance Tracking Tool with the most user friendly Whatsapp like interface.

Comment

Inspiration

Traditional personal finance tools are broken for everyday people. Spreadsheets require tedious manual entry, while most modern budgeting apps overwhelm users with complicated dashboards, rigid input forms, and foreign interfaces. For millions of non-technical users across India and emerging markets, these barriers prevent them from ever building consistent financial tracking habits.

We asked ourselves: What if logging expenses was as natural as sending a WhatsApp voice note to a friend?

People shouldn’t need to adapt to rigid forms or speak fluent English to track their money. They should be able to speak or text in their local dialect, slang, or native language and let an intelligent companion handle the rest. That vision drove the creation of WhatsMyFin: making personal finance accessible, conversational, and effortless for everyone.

What it does

WhatsMyFin is a conversational, mobile-first financial tracking companion built like a familiar chat interface:

  • Conversational Logging (Voice & Text): Users can text or speak casually (e.g., "Spent ₹50 on the boat, ₹60 on the bus, and ₹90 on an auto rickshaw"). The app parses multiple transactions, categorizes them automatically, and presents them cleanly.
  • Human-in-the-Loop Confirmation: Before anything touches the database, the AI presents a clear confirmation card asking for user approval to avoid errors or hallucinated numbers.
  • Dialect & Slang Agnostic: Eliminates language barriers by understanding conversational speech, code-mixed phrasing, and regional slang rather than enforcing rigid English inputs.
  • Automated Analytics & Insights: Tracks cash flow over custom timeframes and surfaces spending patterns without requiring users to build formulas or pivot tables.
  • AI Financial Coaching: Delivers actionable recommendations and summaries to build financial literacy and encourage sustainable spending habits.

How we built it

WhatsMyFin was engineered with an ultra-lean, cost-effective full-stack architecture:

  • Frontend: Built with React to deliver a responsive, mobile-first chat UI.
  • Backend: Powered by FastAPI for high-throughput, low-latency API handling.
  • Database: SQLite for lightweight, persistent local transactional storage.
  • Voice Transcription: Integrated the OpenAI Whisper model to transcribe voice notes locally/efficiently into text before LLM ingestion.
  • AI & Parsing: Utilized the Google Gemini API (via Google AI Studio) strictly for structured JSON extraction and contextual reasoning, keeping credit usage minimal.
  • AI-Assisted Development: Accelerated development using an agentic interview-style prompt workflow inside an AI coding environment. By letting the AI interview us to flesh out requirements before writing boilerplate, we brought roughly 90% of the functional core to life in under 90 minutes.

Challenges we ran into

  • Minimizing API Overhead & Costs: Sending raw audio or large multimodal payloads directly to proprietary LLM endpoints is expensive and consumes excessive API credits. To keep WhatsMyFin sustainable on free-tier access, we decoupled ingestion: audio is pre-transcribed using Whisper, and receipt images are processed through local ML/OCR pipelines before passing only the distilled text to Gemini.
  • Structured Data Reliability: LLMs can be unpredictable with conversational multi-item inputs. We spent significant effort tuning prompts and validation schemas so Gemini reliably returns structured JSON payloads matching our database schema every single time.
  • Balancing Automation with Trust: Users are protective of their money data. We had to ensure the parsing flow didn't silently auto-commit mistakes, designing a frictionless "one-tap approve" card pattern into the chat stream.

Accomplishments that we're proud of

  • Frictionless Entry in Seconds: Completely eliminated the cognitive load of selecting categories, dates, and amounts from dropdown menus. Users can log an entire day's travel in a single 5-second voice note.
  • Ultra-Cost-Efficient Pipeline: Successfully architected the system so that the entire backend operates reliably within the free tier of the Gemini API without sacrificing extraction accuracy.
  • Democratizing Financial Literacy: Created a tool built for the everyday user—bridging the digital divide so anyone comfortable with messaging apps can take control of their financial future.

What we learned

  • Agentic Pair-Programming: We discovered how powerful it is to let an AI coding agent interview the developer. Setting up deep architectural context upfront drastically cut down prompt iterations and code refactoring.
  • Cost-Aware AI Architectures: Learned how to intelligently route tasks—using specialized, lightweight models (Whisper/OCR) for perception and reserving LLMs strictly for structured reasoning.
  • UX for Non-Technical Users: Simple chat interfaces dramatically reduce onboarding friction compared to conventional dashboard-heavy fintech products.

What's next for WhatsMyFin

  • Robust Smart Receipt Scanning: Enhancing local OCR pipelines to instantly itemize paper bills and store receipts straight from a camera snap.
  • Proactive Budgeting Alerts & Limits: Allowing users to set category thresholds (e.g., dining out or food delivery) where the AI proactively warns them in real-time when expenses are trending high.
  • Personalized Financial Coaching & Goal Tracking: Expanding the AI into a supportive financial coach that tracks savings targets and nudges users toward long-term financial health.
  • Privacy-First Local LLMs: Integrating small, quantized open-source models (SLMs) to run directly on-device or low-cost VPS instances, giving users 100% data privacy without dependence on external cloud APIs.

Built With

Share this project:

Updates

Submission history