Trippy

Inspiration

We wanted to make trip planning as simple as swiping through recommendations. Inspired by Tinder’s interface and Netflix’s recommendation system, we combined a swipe interface with ML to personalize travel planning.

What it does

Trippy is a travel recommendation app with a Tinder-like swipe interface. It uses a hybrid ML system (collaborative filtering, content-based filtering, and semantic similarity via Pinecone) to suggest places and activities. Features include: AI Travel Agent: Video consultations with a Tavus AI avatar that extracts preferences from natural conversation Gesture Control: Hand tracking with MediaPipe to swipe through recommendations without touching the screen Multi-User Planning: Group trip planning with recommendations that balance individual preferences Dynamic Content Generation: Uses Gemini AI to generate activities for any destination on demand Confidence-Based Scheduling: Unlocks itinerary building once users reach 95% confidence with 20+ liked items

How we built it

Frontend: Next.js with TypeScript, Framer Motion for swipe animations, and a responsive UI. Backend: Python FastAPI recommendation engine combining: Collaborative filtering (user similarity) Content-based filtering (feature matching) Pinecone semantic search (embeddings) Age-based suitability scoring AI Integration: Tavus AI for video consultations with real-time transcription Gemini AI for generating destination activities Web Speech API for live conversation transcription Gesture Control: MediaPipe hand tracking in Python, integrated via FastAPI endpoints. Database: MongoDB for user data and trip management; JSON-based storage for the recommendation engine.

Challenges we ran into

Combining multiple recommendation algorithms with appropriate weights Real-time gesture detection with low latency and accuracy Extracting structured preferences from natural conversation transcripts Multi-user recommendation balancing individual preferences Dynamic content generation for destinations without pre-existing data

Accomplishments that we're proud of

Hybrid ML system that improves with user interactions Gesture control working reliably for hands-free swiping AI travel agent that feels natural and extracts preferences accurately Multi-user system that generates group-optimized recommendations Confidence threshold system that ensures quality before scheduling

What we learned

Combining multiple ML approaches improves recommendation quality Real-time computer vision requires careful optimization for smooth UX Natural language processing for preference extraction needs robust error handling Age-based scoring with soft multipliers works better than hard filters Group recommendation algorithms need to balance personalization with group consensus

What's next for Trippy

Expand gesture controls to more interactions (pinch to zoom, hand gestures for navigation) Add social features: share trips, collaborative real-time editing Integrate booking APIs for direct reservations Mobile app with native gesture support Advanced scheduling with route optimization and time-based recommendations Personalized travel insights using historical trip data