QuipQuid

From RAGs to riches!

Comment

link to our video

QuipQuid: AI-Powered Budget Tracker

Inspiration

The inspiration for QuipQuid came from recognizing the challenges many people face with managing personal finances. Traditional budget tracking apps often require manual data entry or provide limited insights, making it difficult for users to maintain consistent financial habits. We wanted to create a more intelligent, conversational finance tool that could help users not only track expenses but also gain meaningful insights and make better financial decisions.

What it does

QuipQuid is an AI-powered budget tracking application that combines multimodal input methods with intelligent financial analysis. The key features include:

  • Receipt scanning: Users can take photos of receipts, and the app automatically extracts merchant information, items, and categorizes the expense
  • Split bill functionality: Easily divide expenses among friends and track who owes what
  • Intelligent chat assistant: Ask questions about spending patterns, get budget recommendations, and set financial goals
  • Financial goals tracking: Set and monitor progress toward savings goals with visual feedback
  • Budget management: Create and adjust budgets with AI-generated recommendations
  • Transaction history: View and filter past transactions by category, date, and merchant

Built with

Languages

  • Python (Backend)
  • TypeScript/JavaScript (Frontend)
  • SQL (Database queries)

Frameworks & Libraries

  • FastAPI: Web API framework for Python
  • Next.js: React framework for frontend
  • Tailwind CSS: Utility-first CSS framework
  • ShadCN UI: Component library
  • React Hooks: For state management
  • SQLAlchemy: SQL toolkit and ORM for Python
  • LangGraph: For orchestrating AI workflows
  • Pydantic: Data validation and settings management

Database Systems

  • PostgreSQL: Relational database for structured data
  • MongoDB: NoSQL database for chat history and unstructured data

APIs & Services

  • Anthropic Claude 3.7 Sonnet: For receipt processing and conversation
  • Motor: Asynchronous MongoDB driver

DevOps & Tools

  • Git: Version control
  • Dotenv: Environment variable management
  • Psycopg2: PostgreSQL adapter for Python
  • Uvicorn: ASGI server for FastAPI

How we built it

QuipQuid was built using a modern tech stack with separate frontend and backend components:

Backend Architecture

  • FastAPI: Main web framework for the API endpoints
  • SQLAlchemy + PostgreSQL: For relational data storage
  • MongoDB: For storing chat history and unstructured data
  • Anthropic Claude 3.7 Sonnet: For receipt processing and financial chatbot
  • LangGraph: For orchestrating multi-step AI workflows

Key Backend Files:

  • api.py: Main API endpoints
  • receipt_processor.py: Claude-powered receipt image processing
  • budget_planner.py: Intelligent financial assistant with database connectivity
  • logger.py: Logging system for tracking operations

Frontend Architecture

  • Next.js: React framework for the user interface
  • Tailwind CSS: For styling and UI components
  • ShadCN UI: Component library
  • React Hooks: For state management

Key Frontend Components:

  • FloatingChatBar: Main interface for communicating with the AI assistant
  • GoalsSection: Visual tracking of financial goals
  • DashboardPage: Overview of financial statistics and spending patterns
  • SplitPage: Interface for splitting bills among friends

Challenges we ran into

  1. Receipt processing accuracy: Getting Claude to consistently extract structured data from receipt images required careful prompt engineering and error handling.

  2. Database integration: Creating a system that could both analyze user queries and perform database operations required a sophisticated orchestration layer.

  3. Reactive UI design: Ensuring the app felt responsive and intuitive on both mobile and desktop platforms while maintaining a cohesive design language.

  4. Handling multiple data sources: Integrating various data sources (images, text, database) into a coherent system required careful architecture planning.

  5. Optimizing AI response time: Balancing the need for detailed financial analysis with response speed to maintain a good user experience.

Accomplishments that we're proud of

  1. Seamless multimodal input: Successfully implemented a system that can process different types of inputs (text, images) and translate them into structured financial data.

  2. Intelligent financial assistant: Created a conversational interface that not only answers questions but can also perform complex financial analysis and make personalized recommendations.

  3. Robust workflow system: Implemented LangGraph to create reliable, multi-step AI workflows that handle various user intents.

  4. Modern, responsive UI: Designed an intuitive interface that works well across all device sizes while maintaining visual appeal and functionality.

  5. Privacy-preserving architecture: Built the system to process sensitive financial information securely.

What we learned

  1. AI orchestration: Gained expertise in creating complex, multi-step AI workflows that can handle different user intents and edge cases.

  2. Prompt engineering: Learned techniques for crafting effective prompts that extract structured data from unstructured inputs like receipt images.

  3. Multimodal systems: Developed skills in building applications that seamlessly integrate different input modalities.

  4. Financial data modeling: Deepened our understanding of how to model financial data in a way that enables meaningful analysis and insights.

  5. User-centered design for financial apps: Learned important principles about making financial applications accessible and engaging.

What's next for QuipQuid

  1. Bank integration: Connect directly with banks and credit cards to automatically import transactions.

  2. Advanced financial insights: Implement more sophisticated analysis and forecasting capabilities.

  3. Social features: Enhance the bill-splitting functionality with social elements like group expense tracking.

  4. Investment tracking: Add support for monitoring investments and portfolio performance.

  5. Personalized financial education: Provide tailored educational content based on users' specific financial situations and goals.

  6. Multiple language support: Expand the application to support users around the world in their native languages.

  7. Offline functionality: Improve the app to work seamlessly in low-connectivity environments.

Built With

Share this project:

Updates