About this project
Inspiration
As a CPA, I was inspired to create an AI assistant that could provide real-time, accurate tax advice while making complex tax information accessible to everyone. Traditional tax software is often expensive and static, while free resources can be outdated or unreliable. I wanted to build something that combines the expertise of a professional CPA with the convenience of modern AI technology, making tax knowledge available 24/7 through natural conversation.
What it does
xBav.ai is an intelligent CPA assistant that provides real-time tax advice and document analysis through voice conversation. The app features:
- Voice-First Interface: Natural voice conversations with AI using ElevenLabs text-to-speech and speech-to-text
- Real-Time Tax Information: Web search integration for current tax laws, deductions, and IRS updates
- Document Intelligence: Upload tax documents (W-2s, 1099s, receipts) for AI-powered analysis and context
- RAG-Powered Responses: Combines uploaded documents with real-time web search for personalized, accurate advice
- Professional CPA Expertise: Built with comprehensive tax knowledge and best practices
Users can ask questions like "What are the 2025 standard deductions?" or "How much tax will I owe on $100,000?" and get immediate, accurate responses with current information.
How we built it
Frontend: React Native with Expo for cross-platform mobile development
- Custom voice recording interface with animated feedback
- Document upload and management system
- Real-time chat interface with voice playback
Backend: Supabase for database and serverless functions
- PostgreSQL database with vector embeddings for document search
- Edge Functions for secure API handling and RAG implementation
- Real-time document processing with Google Document AI
AI Integration:
- OpenAI GPT-4o for intelligent responses and document analysis
- ElevenLabs for natural voice synthesis and speech recognition
- Google Custom Search API for real-time tax information
- Vector similarity search for document retrieval
Key Technologies:
- Supabase (Database, Auth, Storage, Edge Functions)
- React Native/Expo
- OpenAI API (GPT-4o, Embeddings)
- ElevenLabs Voice API
- Google Document AI & Custom Search API
- PostgreSQL with pgvector extension
Challenges we ran into
Voice Integration Complexity: Initially struggled with ElevenLabs SDK limitations and WebSocket connection stability. Solved by creating a Supabase Edge Function for secure API key handling and implementing proper WebSocket lifecycle management.
Document Processing: PDF parsing was unreliable with basic text extraction. Switched to Google Document AI for robust PDF processing and implemented text cleaning to handle null characters and formatting issues.
Real-Time Information: Needed current tax data but faced limitations with static knowledge bases. Implemented intelligent web search with multiple query strategies and comprehensive fallback systems.
Token Management: AI responses were getting cut off due to token limits. Developed a concise response strategy that provides complete, actionable information within constraints.
Vector Search Setup: Initial challenges with pgvector extension and embedding generation. Iteratively improved the RAG system with better chunking strategies and similarity thresholds.
Accomplishments that we're proud of
- Complete Voice-First Experience: Successfully integrated natural voice conversation with professional-grade text-to-speech and speech recognition
- Real-Time Tax Intelligence: Built a system that provides current, accurate tax information by combining web search with AI analysis
- Document Intelligence: Created a robust document processing pipeline that can analyze uploaded tax documents and provide contextual advice
- Professional-Grade Accuracy: Achieved high accuracy in tax calculations and advice through comprehensive knowledge base and real-time verification
- User-Friendly Interface: Designed an intuitive mobile app that makes complex tax information accessible through natural conversation
- Scalable Architecture: Built a serverless system that can handle multiple users and document types efficiently
What we learned
AI Integration Best Practices: Learned how to effectively combine multiple AI services (OpenAI, ElevenLabs, Google APIs) for a cohesive user experience.
Voice Technology: Gained deep understanding of voice processing, including speech-to-text, text-to-speech, and voice activity detection.
RAG Implementation: Mastered retrieval-augmented generation techniques for combining document context with real-time information.
Edge Computing: Learned to build efficient serverless functions for handling sensitive API keys and processing user data securely.
Tax Domain Expertise: Developed comprehensive understanding of tax information architecture and how to make it accessible through AI.
Mobile Development: Gained experience with React Native, Expo, and mobile-specific considerations like audio handling and file uploads.
What's next for xBav.ai
Enhanced Features:
- Multi-language support for diverse user base
- Integration with tax filing software (TurboTax, H&R Block APIs)
- Advanced document analysis for complex tax scenarios
- Voice biometrics for secure user authentication
Expansion Opportunities:
- Business tax planning and consultation features
- Integration with accounting software (QuickBooks, Xero)
- Real-time tax law change notifications
- Personalized tax planning recommendations
Platform Growth:
- Web application for desktop users
- API for third-party integrations
- Partnership with accounting firms for professional use
- Educational platform for tax learning and certification
AI Improvements:
- Fine-tuned models for specific tax scenarios
- Advanced reasoning for complex tax situations
- Predictive analytics for tax planning
- Continuous learning from user interactions
The goal is to make xBav.ai the go-to platform for intelligent tax assistance, bridging the gap between professional CPA services and accessible, affordable tax help for everyone.
Log in or sign up for Devpost to join the conversation.