VibeScanner

Inspiration

We've all scrolled through photos or listened to a song that makes us feel somewhere. The problem is that every travel app makes you know where you want to go before you start searching.

We wanted to flip that. What if you could just show the app what's your vibe, and it figures out the destination?

What it does

VibeScanner is web application that recommends travel destinations based on images or music. Discover places that match your vibe, whether it's a photo you love or a song you're obsessed with.

Image-Based Recommendations

• Upload one or more photos
• Select your desired travel month
• The app analyzes the "vibe" of your images
• Get destination suggestions that match the mood and aesthetics

Music-Based Recommendations

• Enter an artist and a song
• Musical tags are assigned to the song (e.g., genre, mood, energy, etc.)
• Receive travel destination suggestions alignes with the song's vibe

How we built it

Frontend

• Next.js (React)
• Tailwind CSS
• TypeScript

Backend

• FastAPI (Python)

AI Models

• Ollama
  • Llava: for image analysis
  • Llama-3.2-1B: for text interpretation

External APIs

• Skyscanner API: flights
• Last.fm: music tags

Challenges we ran into

Using Ollama with the Llava model pushed hardware to its limits. We had to lower resource usage to make it stable.

Accomplishments that we're proud of

• Building a functional application integrating multimodal inputs into one system
• Designing an intuitive user experience
• Using Ollama model Llava, which we had never used before.

What we learned

• Working with AI Models for both images and text using Ollama
• The importance of prompt design when working with language models
• How to integrate the APIs used

What's next for VibeScanner

• We would like to replace Ollama with a cloud-based model such as GPT.4o, Claude 3.5 Sonnet or open-source alternatives.
• To be able to connect your social media so it analyzes your general vibe.

Built With

• fastapi
• last.fm
• llama3.2:1b
• llava
• next.js
• ollama
• skyscannerapi
• tailwindcss
• typescript