Receiptly

Inspiration

Managing daily expenses manually — especially from paper receipts — is tedious, error-prone, and easy to ignore. Many people (including us) tend to stuff receipts into wallets or drawers, only to forget them entirely. We wanted to simplify this common pain point by building a tool that automates expense tracking straight from receipt images.

With advancements in OCR (Optical Character Recognition) and NLP (Natural Language Processing), we saw an opportunity to bridge this gap: turn raw receipt images into meaningful, itemized financial data with minimal user effort.

Our goal was to build something that:

  • Helps users stay on top of their personal finances
  • Eliminates manual data entry
  • Provides clear insights through interactive charts and dashboards
  • Works from a simple web interface

We were also curious about applying AI to real-world use cases — this project allowed us to integrate machine learning, backend services, and frontend design into a meaningful and usable product.

What it does

Receiptly is a full-stack web application that allows users to upload images of their receipts or bills. It processes these images using OCR and NLP to extract item-level details such as name, price, quantity, and category.

The extracted data is:

  • Stored securely in MongoDB
  • Displayed through interactive dashboards
  • Used to help users track monthly spending, categorize expenses, and plan budgets

Key Features

  • Upload receipt images directly from the web UI
  • Extract structured data using Python OCR & NLP
  • Store data securely in MongoDB
  • Track monthly expenses and budget status
  • View receipt history with itemized details
  • Generate interactive charts for spending insights
  • Integrated calendar to view bills by receipt date
  • Chatbot for personalized spending suggestions
  • Budget planner for each month
  • User authentication

How we built it

Receiptly combines multiple technologies into a unified system:

  • Frontend: React.js + Bootstrap for a responsive and clean UI
  • Backend: Node.js + Express.js handling API requests and user interactions
  • OCR + NLP: Python (Tesseract / pytesseract, spaCy) to extract and process text from receipt images
  • Database: MongoDB (local for now, with plans to shift to cloud-hosted)
  • Charts: Chart.js / Recharts for dynamic visualization

Challenges we ran into

  1. OCR accuracy: Receipts with poor print quality or unusual layouts caused extraction errors, requiring custom cleaning logic.
  2. NLP tuning: Designing rules to correctly categorize and structure items from the raw text was tricky and required iteration.
  3. Integrating frontend + backend + Python scripts: Handling seamless communication between the Node server and Python processing scripts needed careful design.

Accomplishments that we're proud of

  • Successfully processing receipts with a reasonable accuracy for item detection
  • Building a clean, user-friendly interface for uploading and tracking receipts
  • Creating interactive charts and summaries to make data meaningful
  • Integrating multiple technologies into a single working system

What we learned

Deepened understanding of OCR/NLP and their real-world quirks
Learned to structure full-stack apps with separate microservices
Gained practical experience with Docker and cloud deployment prep
Improved skills in frontend design and interactive data visualization

What's next for Receiptly

  • Deploy the app fully on Google Cloud or another platform
  • Add user profiles
  • Enhance chatbot suggestions using a fine-tuned LLM
  • Improve OCR accuracy with custom-trained models
  • Add export options (PDF, CSV reports)
Share this project:

Updates