OExpenso

Voice-Activated Smart Expense Tracker

Comment

🚀 Inspiration

Manually entering expenses is tedious and often gets skipped. We wanted to simplify this by creating a hands-free solution that lets users log expenses on the go, just by speaking. The idea was to make personal finance management feel as natural as having a conversation.

💡 What it does

OExpenso is a voice-activated smart expense tracker.
Users speak their expenses (e.g., "Spent ₹500 on groceries"), and the app:

  • Converts speech to text using ElevenLabs API
  • Sends the text to OpenAI’s LLM for data extraction
  • Extracts structured data like Amount, Category, and Description
  • Saves transaction details in Supabase
  • Updates the Analytics and History tabs in real time

The Analytics tab gives users insights into their spending patterns, while the History tab allows editing or reviewing past entries.

🛠️ How I built it

  • React Native for cross-platform mobile development
  • ElevenLabs API for high-quality speech-to-text transcription
  • OpenAI GPT API for parsing natural language and extracting structured data
  • Supabase for storing, retrieving, and managing all transaction data
  • Chart.js for visualizing spending analytics

🧩 Challenges I ran into

  • Accurately handling diverse and casual speech (e.g., "had lunch with 3 friends for ₹900")
  • Ensuring low-latency voice processing to maintain a fast user experience
  • Extracting structured data like category and date reliably using LLMs
  • Syncing data across different app states without lag

🏆 Accomplishments that I'm proud of

  • Built a working end-to-end voice-to-expense pipeline
  • Designed a clean, intuitive UI with live analytics and editable history
  • Successfully integrated LLM-based NLP to make expense logging conversational
  • Enabled real-time updates and editable entries using Supabase

📚 What I learned

  • Voice-based UX requires careful design and immediate feedback to build trust
  • LLMs are powerful for extracting structured data from casual, varied user input
  • User context (e.g., "spent with 2 people") adds complexity and nuance in expense extraction
  • Clear UI/UX helps guide users through AI-powered flows

🔮 What's next for OExpenso

  • Offline voice processing using on-device models for better privacy
  • Multi-user/shared wallets for families or roommates
  • Recurring expense reminders and budget goals
  • Custom category training based on user habits
  • Exporting data to spreadsheets or finance tools (e.g., Notion, Google Sheets)

Built With

Share this project:

Updates