Inspiration
As 2nd-year B.Tech students, we often felt a step behind. We watched peers build amazing tools while we struggled to find our footing. After failing to make an impact at our local college hackathon due to a lack of technical direction, we made a pact: we wouldn't just learn theory; we would build. This Code Spring Hackathon became our target—a beginner-friendly arena to prove to ourselves that we could deliver.
The specific idea for MoodDJ was born from a personal frustration. I love music, but I suffer from "choice paralysis." I would spend more time skipping tracks to match my current headspace than actually listening, often ending up frustrated or listening to the same stale playlist. We asked ourselves: Why can't the music player just "know" how we feel? That question kickstarted our journey to build an empathetic AI that listens to your emotions before you listen to the music.
What it does
MoodDJ is an AI-powered, context-aware music ecosystem that bridges the gap between human emotion and digital experience. It doesn't just shuffle songs; it curates a vibe.
👁️ Real-Time Emotion Recognition: Using a webcam and the RMN (Residual Masking Network) architecture, the app detects the user's emotional state (e.g., Happy, Sad, Neutral, Angry) with high accuracy.
🧠 Context-Aware Curation: It isn't limited to just facial expressions. The system analyzes local weather and time of day to refine recommendations (e.g., playing "Lo-fi beats" for a rainy evening vs. "High energy pop" for a sunny morning).
🎧 Dynamic Audio Engine: Instead of relying on restricted 30-second clips, we built a custom audio engine using yt-dlp to fetch and play full audio dynamically from YouTube based on AI suggestions.
📊 Emotional Analytics: The "Insights Dashboard" visualizes the user's emotional history using Radar charts (Emotional Fingerprint) and Area charts (Vibe Flow), helping users understand their mood trends over time.
How we built it
Our development process was an evolution, moving from a simple script to a complex full-stack architecture.
Phase 1: The Prototype (Python & Streamlit) As beginners, we started with what we knew. We built a lightweight MVP using Streamlit for the UI and OpenCV for the camera feed. We used DeepFace for basic detection and hardcoded Spotify links embedded into the app. This proved the concept worked but highlighted severe limitations in user experience and customization.
Phase 2: The Pivot (React & FastAPI) To create a premium "Arctic Minimalist" experience, we completely overhauled the stack:
Backend: We migrated to FastAPI for high-performance, asynchronous handling. We replaced hardcoded logic with Groq (Llama-3), allowing an LLM to curate songs intelligently based on the detected mood + weather + time.
Frontend: We moved to React 18 (Vite). This allowed us to implement complex state management for the music player and use Three.js for immersive 3D particle animations that react to the music.
Real-Time Data: We implemented WebSockets to handle the video feed and emotion data transmission with low latency, ensuring the music reacts instantly to face changes.
Challenges we ran into
This project was a series of technical hurdles that forced us to learn fast:
The Spotify Embed Wall: Initially, we relied on the Spotify SDK. We quickly realized it was too restrictive it didn't allow programmatic playback control without premium auth tokens and often blocked specific songs. This threatened to kill the project.
Solution: We pivoted to a custom backend using yt-dlp, which searches and streams audio from YouTube, giving us total control over the playback engine.
The "Ghosting" Bug: In our early Streamlit prototype, the webcam feed would freeze or "ghost" when the UI updated.
Solution: This forced our migration to React, where we could separate the video rendering thread from the UI logic.
Language Limitations: We tried using the Jamendo API as a middle ground, but it lacked regional content (Hindi/Tamil songs), which was personal to us.
Solution: Integrating the LLM (Llama-3) allowed us to request songs in any language or genre naturally.
Accomplishments that we're proud of
From Zero to Full-Stack: In just two weeks, we went from "no idea how to approach this" to building a fully functional app with a separate frontend, backend, and AI service.
Custom Audio Engine: We are proud that we didn't just paste a Spotify widget; we built our own player that handles audio streams, likes, and history.
The UI/UX: Achieving the "Spotify-inspired" dark mode with 3D particle animations (using Framer Motion and Three.js) makes the app feel professional and polished, far beyond our initial expectations.
What we learned
This hackathon was a crash course in modern software engineering.
System Architecture: We learned how to connect a React frontend to a Python backend using REST APIs and WebSockets.
AI Integration: We gained a deep understanding of how to implement Computer Vision models (RMN) and how to prompt-engineer LLMs (Groq) to get structured JSON data for our app.
Resilience: The biggest lesson was that "Plan A" rarely works. Rewriting our entire codebase halfway through because the tech stack wasn't scaling was terrifying, but it was the best decision we made. We learned to embrace the pivot.
What's next for CodeSpringers
This hackathon was just the beginning of our journey. We are committed to carrying this momentum forward by participating in upcoming hackathons and constantly challenging ourselves with new problems. Our goal is to keep expanding our technical toolkit, refining our development skills, and turning more innovative ideas into reality. We are eager to learn, build, and grow even further.
Log in or sign up for Devpost to join the conversation.