PinVision

Turn your Pinterest board into an interactive, shoppable event visualization, powered by Gemini AI.

Comment

Inspiration

Everyone has a Pinterest board full of dream aesthetics: beautiful party setups, color palettes, and vibes they love. But turning that inspiration into an actual event plan is overwhelming. We asked: what if your Pinterest board could
tell you exactly what your aesthetic is, show it to you as data, and then build your dream party scene with shoppable recommendations, all automatically?

What it does

PinVision transforms any Pinterest profile into a fully interactive event planning experience:

  1. Paste your Pinterest URL: PinVision scrapes all your public pins automatically
  2. Gemini AI analyzes your aesthetic: every image is analyzed using the Gemini Vision API for colors, moods, objects, and style, producing a rich data profile of your taste
  3. Explore your aesthetic dashboard: an immersive data visualization experience (built for Peraton's Best Data Visualization track) shows your dominant colors, mood breakdown, style clusters, object co-occurrence networks, and brightness/saturation scatter plots across your entire Pinterest history
  4. Click your dream party scene:a FLUX.1-schnell generated scene based on YOUR aesthetic has 7 clickable elements. Click any object and a sidebar slides in with real product recommendations and DIY guides, all tailored to your
    style, powered by Gemini

How we built it

Team: Rahul Chithra Shiva, Swetha Krishna Avula, Dhanya Sri Cherukuri

AI Pipeline:

  • pinterest-dl scrapes all public pins from a Pinterest profile
  • Gemini Vision API (gemini-2.0-flash) analyzes every image, extracting colors, mood labels, style tags, party relevance, and object lists across 218 images
  • Gemini aggregates the analysis into a structured aesthetic profile and generates an optimized prompt capturing the user's exact style
  • FLUX.1-schnell generates a photorealistic party scene tailored to the user's aesthetic
  • Gemini generates personalized product recommendations and DIY guides for each clickable scene element

Data Visualization Dashboard:

  • Built with React + Recharts
  • 8+ chart types: Pie chart, Bar chart, Treemap, Radar chart, Cluster map (t-SNE), Mood x Color heatmap, Co-occurrence network graph, Brightness/Saturation scatter plot
  • Interactive party scene with SVG polygon overlays and animated sidebar

Backend API:

  • FastAPI with Pydantic models serving all pre-cached data

Challenges we ran into

  • Gemini rate limits: 15 RPM on the free tier meant batching 218 images carefully with incremental saves so no progress was lost
  • Scene segmentation: We attempted LangSAM, Gemini Vision polygon tracing, GPT-4o tracing, Replicate API, and SAM 2.1 but all produced unreliable results. We shipped hand-tuned bounding boxes that work reliably for the demo
  • Three-way parallel development: Scripts, backend, and frontend were built simultaneously by three people against a shared JSON schema with no live backend during development. We solved this with strict directory ownership and a mirrored local data folder

Accomplishments that we're proud of

  • Gemini powers the entire intelligence layer: 3 distinct Gemini API calls handle vision analysis of 218 images, aesthetic profile synthesis, and personalized product/DIY recommendations
  • 8+ interactive visualizations all driven by real scraped Pinterest data, not dummy data
  • The interactive party scene where clicking a balloon cluster shows product recommendations matched to YOUR color palette
  • Built a full end-to-end AI product in under 24 hours by a team of three

What we learned

  • How to orchestrate multiple Gemini API calls in a real pipeline: vision, structured extraction, and generative recommendations
  • How to design a shared JSON schema that three people can build against independently without conflicts
  • How to turn raw scraped social media data into meaningful, beautiful data visualizations
  • That t-SNE clustering on image aesthetic features actually groups Pinterest pins into surprisingly coherent style clusters

What's next for PinVision

  • Real-time processing: Run the full pipeline live instead of overnight caching
  • Multi-board support: Analyze specific boards (weddings, birthdays, home decor) separately
  • Budget filtering: Filter product recommendations by price range
  • Vendor integration: Direct links to Etsy, Amazon, and local vendors matching the user's exact aesthetic

Built With

Share this project:

Updates