🖐️ FinanceAssist – Project Story

🌟 Inspiration

Money touches every part of life, yet managing it often feels chaotic. We found ourselves juggling:

Scattered receipts
Unexpected subscription renewals
Confusing spending patterns
Manual transactions we kept forgetting to log
Multiple apps that never worked together

That’s when we asked:

“What if managing your money felt as simple as chatting with a friend?” “What if your finances could organize themselves?”

FinanceAssist was born from this vision — an AI-powered personal finance assistant that listens, analyzes, predicts, and helps you take control effortlessly.

🧠 What It Does

FinanceAssist is a multi-modal, AI-powered personal finance assistant designed to simplify how users track and understand their finances. It combines automation, intelligent analysis, and a beautiful interface to deliver a seamless financial experience.

⭐ 1. Smart Dashboard — Complete Financial Overview

FinanceAssist’s dashboard provides an instant snapshot of a user’s financial world:

Total spent this month
Predicted cash flow for upcoming periods
Financial health score (based on savings, stability, subscriptions)
Upcoming bill renewals
Biggest spending categories
Visual analytics: line charts, donut charts, bar charts
Quick Action

Every widget updates in real time as new data flows in.

⭐ 2. Transaction Tracking (Manual + Voice + NLP)

FinanceAssist lets users record expenses in three intuitive ways:

✔ Manual Entry

Straightforward fields for:

Description
Amount
Date
Category

✔ Speech-to-Text Entry

“Paid 350 for groceries yesterday.” → Automatically categorized as Food & Dining.

✔ Auto-Categorization (NLP)

FinanceAssist uses keyword detection + ML-based NLP to classify transactions into categories like:

Food
Shopping
Transport
Bills
Entertainment
Others

The system becomes smarter as more data is added.

⭐ 3. Receipt Scanner (OCR + Item Extraction)

FinanceAssist extracts structured data from uploaded JPG, PNG, and PDF receipts using TAGGUN OCR.

It identifies:

Store name
Total amount
Tax
Date
Itemized list (if available)

✔ Manual Correction

Users can fine-tune:

Items
Tax
Date
Category
Amount

Ensuring high accuracy even for imperfect receipts.

⭐ 4. Shared Bill Splitting (with WhatsApp Sharing)

A fully implemented, user-ready feature.

If a receipt contains multiple items, FinanceAssist allows users to:

Select the items they personally consumed
Automatically calculate:
- Their subtotal
- Proportional tax amount
- Final payable total

✔ WhatsApp-Sharable Summary (Completed)

Once the split is done, FinanceAssist generates a beautifully formatted, ready-to-send WhatsApp message, including:

Itemized list
Tax & subtotal
Final owed amount
Clean and friendly formatting

Users can instantly share the final bill with friends — WhatsApp sharing applies to this Shared Bill Splitting feature.

⭐ 5. Subscription Tracker — Know Your Renewals

Users can add and manage recurring subscription expenses with:

Subscription name
Monthly cost
Renewal date
Category

FinanceAssist highlights renewals 7 days before their due date and displays subscription analytics such as:

Total monthly subscription cost
Costliest subscriptions
Yearly projections

This helps users avoid surprise charges.

⭐ 6. AI Financial Assistant (RAG-Powered)

FinanceAssist includes a conversational AI that analyzes real user data to answer questions like:

“How much did I spend on transportation this week?”
“What categories am I overspending on?”
“Forecast my cash flow next month.”

This transforms FinanceAssist into a personal financial advisor that is always available.

FinanceAssist doesn’t just track your money — it understands it.

🛠️ How we built it

We designed FinanceAssist as a scalable, AI-first, cloud-native system.

🔧 Tech Stack

Frontend: Next.js, Tailwind CSS
Backend: FastAPI (Python)
Database: TiDB
AI Features: RAG, NLP, Claude API
OCR: TAGGUN API
Speech AI: ElevenLabs Speech-to-Text
Platform: Cursor

We engineered FinanceAssist for speed, simplicity, and intelligence — with each component seamlessly connected.

🧗 Challenges we ran into

Every feature pushed us into new territory:

OCR variability — receipts come in every possible format
NLP categorization for messy, natural language transaction descriptions
Designing a bill-splitting model that correctly allocates TAX and subtotals
Building a fast RAG pipeline that returns insights within seconds
UI/UX balancing — presenting deep financial insights without overwhelming the user

Each challenge forced us to rethink, redesign, and ultimately build a more powerful system.