Pinlist

Everyone associates experiences and aesthetics with music. Pinlist analyzes visual moods from your Pinterest board or camera and map it into personalized playlist to match your vibe.

Comment

Inspiration/ Abstract

The Market Opportunity

  • In 2026, social media platforms such as Instagram and TikTok have been a main source of music discovery for the younger generation, Gen Z.
  • Despite being a top-tier platform for Gen Z inspiration, Pinterest has historically lacked the short-form audio integration that drives 75% of music discovery on TikTok and Instagram (MRC Data).
  • Pinlist aims to fill in this market gap, allowing users to translate highly aestheticized visuals (Pinterest Boards) into personalized music recommendations (Spotify Playlists).
  • Not only a music discovery platform, Pinlist serves as a social media app for users to share their connected visual/audio aesthetics via Airbuds/ Spotify Wrapped inspired cards to a short form content styled For You page.

What it does

  • pins == pinterest images

  • board == collage of pins

  • Multimodal Aesthetic Analysis: Uses an advanced CLIP (Contrastive Language-Image Pre-Training) model to classify visual content across 20+ specific mood labels, ranging from "Old Money Mediterranean" to "Dark Academia" and "Gothic."

  • Intelligent Music Mapping: Translates high-dimensional image embeddings into Spotify-ready audio features (danceability, energy, valence) to query a custom recommendation engine.

  • Interactive Explore Scroll: A mobile-optimized, TikTok-style vertical feed where users can swipe through aesthetic cards with auto-playing audio previews and immersive glassmorphic UI.

  • Board & Photo Modes:

    • Board Mode: Aggregates hundreds of pins to create a "Mood Fingerprint" of an entire Pinterest presence.
    • Photo Mode: Analyze local uploads or "fit pics" for instant, one-off vibe checks without requiring a Pinterest board.

  • Visual Vibe Cards: Generates high-fidelity shareable assets featuring a curated collage of pins, track metadata, and dynamic mood badges.

How we built it

Modeling

  • CLIP-Powered Vision Pipeline: Built using transformers and PyTorch, the system runs every image through an openai/clip-vit-base-patch32 model to compute similarity scores against niche aesthetic labels.
  • Mood Fingerprinting: Developed a custom algorithm that resolves subcategories (e.g., "wired earbuds" -> "performative") to provide a stabilized top-3 mood profile for every board.

Backend

  • Flask & SQLite Orchestration: A robust Python backend manages the heavy lifting, handling image processing, DB persistence for the Explore feed, and secure API routing.
  • Token-Exchange Architecture: Implemented a secure session management system (X-Session-ID) to handle Pinterest and Spotify OAuth without exposing raw access tokens to the frontend.
  • In-Memory Token Store: Integrated a fast lookup system for active sessions to ensure low-latency requests during high-intensity image analysis.

Frontend

  • Vanilla JS + Modern CSS: Created a high-end, responsive UI that mirrors Pinterest’s design language while introducing ambient "dark mode" aesthetics.
  • Intersection Observer Engine: The Explore Scroll uses the IntersectionObserver API to manage resource-heavy audio previews and animations, ensuring smooth performance during infinite scrolling.

Challenges we ran into

  • CLIP Inference Performance: Running deep learning models on every pin in a massive board was initially slow. We optimized the pipeline using batch processing and stabilized the "Mood Fingerprint" through weighted probability distributions.
  • The Auth Mesh: Coordinating two different OAuth flows (Pinterest + Spotify) without forcing the user into a loop of redirects. We solved this by creating a centralized auth.js helper that manages persistence via a unique session ID.
  • Screenshot Logic: Generating an image from a live DOM element is notoriously buggy. We innovated by creating a "Clean Stage" screenshot overlay, a dedicated UI state that strips away navigation, allowing users to take a perfect native screenshot of their card.

Accomplishments that we're proud of

  • Zero-Shot Aesthetic Recognition: Successfully implemented a CLIP model that understands niche internet aesthetics (like "Performative Matcha" or "Coastal Beach") far better than standard image tagging.
  • The Global Feed: Built a fully functional social layer where users can save their "Vibe Cards" to a shared database for others to discover.
  • Multi-Platform Integration: Orchestrated a complex multi-platform login (Pinterest + Spotify) that feels unified, secure, and seamless.
  • Dynamic UI: Achieved a professional-grade "look and feel" using modern glassmorphism and real-time rendering of masonry layouts.

What we learned

  • Advanced AI Architectures: Deep dive into CLIP and Transformer-based models for zero-shot image classification.
  • Multimodal Engineering: How to map visual "feel" (color, contrast, density) to mathematical audio parameters (valence, tempo).
  • Secure Session Handling: Learning how to protect user data by using a "middleman" Flask server to hide sensitive OAuth tokens.
  • Optimization: Implementing lazy-loading, intersection observers, and batch processing for high-performance web AI tools.

What's next for Pinlist

  • Native Mobile App: Porting the high-speed Explore Scroll into a native iOS/Android experience using React Native.
  • Enhanced GNN Logic: Implement a Graph Neural Network to analyze the relationships between pins to recommend even more niche, curated tracklists.

Built With

Share this project:

Updates