Songsnap

Turn your photo memories into musical blasts from the past! 🎵📷

Comment

Inspiration 💡

The inspiration behind Songsnap came from the desire to combine the nostalgia of photos with the emotional resonance of music. We wanted to create a tool that could analyze the visual elements of a photo and curate a Spotify playlist that captures the essence of the scene depicted.

What it does 🎵

Songsnap utilizes GPT-4 Vision to extract detailed information from images such as time period, mood, cultural elements, and location. This data is then summarized using Cohere into concise bullet points. Users can upload photos to the platform, which then generates a curated playlist based on the image description using a fine-tuned GPT-3.5 model. The generated playlists can be accessed and played directly through the website thanks to integration with the Spotify API. Additionally, users can manage and access their playlists through a library page powered by Supabase, a PostgreSQL database API. We also have an Auth0 login system set up.

How we built it 🔨

We built Songsnap using a combination of advanced AI models, APIs, and web development technologies. Here's the breakdown:

  • Image Analysis: GPT-4 Vision for extracting image information.
  • Summarization: Cohere for summarizing image data into bullet points.
  • Playlist Generation: Fine-tuned GPT-3.5 model to generate playlists based on image descriptions.
  • Web Development: Flask framework for building the website.
  • Authentication: Auth0 for user authentication.
  • Spotify Integration: Spotify API for playlist generation and playback.
  • Database Management: Supabase for storing and managing user playlists using PostgreSQL.

Challenges we ran into 🚩

  • Model Integration: Integrating multiple AI models and APIs seamlessly was challenging and required thorough testing and debugging.
  • Data Handling: Managing and processing large amounts of image data and playlist information efficiently was a significant challenge.
  • API Limitations: Working within the limitations of third-party APIs, especially Spotify and Auth0, posed challenges during development and integration.
  • User Experience: Ensuring a smooth and intuitive user experience across the website, especially with playlist management and playback, required careful design and implementation.

Accomplishments that we're proud of ⭐

  • Successfully integrating multiple AI models and APIs to create a cohesive platform.
  • Building a user-friendly website with seamless authentication and playlist management features.
  • Generating accurate and relevant playlists based on image descriptions using AI-driven analysis.
  • Overcoming technical challenges and delivering a functional prototype within the hackathon timeframe.

What we learned 📚

  • Deepened our understanding of AI model integration and API usage.
  • Enhanced our skills in web development, particularly with Flask and database management.
  • Learned effective strategies for handling and processing diverse data types, including images and text.
  • Gained insights into user experience design and optimization for web applications.

What's next for Songsnap 🚀

  • Enhanced Image Analysis: Further refining the image analysis capabilities to extract even more detailed and nuanced information.
  • Improved Playlist Generation: Fine-tuning the playlist generation algorithm to provide more accurate and diverse song recommendations.
  • Community Engagement: Implementing social features to allow users to share and discover playlists created by others.
  • Mobile App Development: Expanding Songsnap's reach by developing mobile applications for iOS and Android platforms.
  • Pitching: Seeing if Spotify is interested in the idea.

Built With

Share this project:

Updates