🕺 ChoreoRockstar: AI Dance Grader
“Where technology meets rhythm — and every move becomes data-driven art.” 🎶💫
🎯 Inspiration
Every dancer dreams of improving — but not everyone has access to professional feedback.
ChoreoRockstar was born from that spark 💡: What if an AI could act as your personal dance judge?
A system that watches your moves, feels your rhythm, and scores your artistry like a real competition — blending creativity, technology, and passion into one neon-lit experience. 🌟🎶
💡 What It Does
ChoreoRockstar is an AI-powered web app that transforms your performances into AI-evaluated choreography reports.
Upload a photo or short video, and our AI judge “AI-lex” analyzes your technique, rhythm, and style — giving you:
- 🏆 Performance Score
- 🎨 Creativity & Expression Analysis
- 🎵 Musicality & Flow Feedback
- ⭐ Rockstar Tip — a personalized improvement insight
If you score above 85, you unlock a Rockstar Splash — a confetti explosion celebrating your excellence. 🎉🤘
🎥 ChoreoRockstar Gallery
⚙️ How We Built It
- 🧠 AI Core: Google Gemini 2.5 Pro (multimodal) interprets movement, emotion, and sound.
- 🖼️ Input Processing: Client-side extraction of keyframes & audio to ensure privacy.
- 💬 AI Persona: “AI-lex” — a sassy, professional dance judge who never sleeps but always vibes.
- 💻 Frontend: Next.js + React, styled with Glassmorphism + neon cyber-rock aesthetic.
- 🧩 Backend: Node.js + Express with secure API routing.
- ☁️ Cloud Infrastructure: Fully containerized and deployed via Google Cloud Run, powered by Vertex AI and Firebase Hosting.
🧠 Advanced Integrations
- 🎧 ElevenLabs — AI voice synthesis for spoken feedback (“AI-lex Speaks” 🎤).
- ⚙️ LiquidMetal — GPU optimization layer for real-time inference rendering.
- ☁️ Raindrop — Data stream orchestration for seamless cloud data sync.
- 🧮 Cerebras Cloud — Parallel neural acceleration for visual inference.
- 🤗 Hugging Face Transformers — Pretrained models for movement and emotion recognition.
- 🧠 Replicate — Model deployment for choreography segmentation prototypes.
- 🔊 Google Cloud Speech + Vision APIs — For rhythm detection and motion focus.
- 🔥 Firebase — Authentication, analytics, and user session tracking.
- 🗂️ Google Cloud Storage (GCS) — Temporary video asset storage.
- 🧰 Vertex AI + Cloud Run — Custom multimodal model endpoints with scalable inference.
- 🌐 Vercel — Frontend hosting for global latency optimization.
- 🛠️ Render + Railway — Redundant backend deployment & testing environment.
🎯 Purpose
ChoreoRockstar is a web application that acts as your personal AI dance coach and judge.
It analyzes dance performances through images or videos using advanced multimodal AI and delivers personalized, data-driven feedback — all wrapped in a dark, futuristic, rockstar-inspired aesthetic with neon cyan and fuchsia highlights. ⚡🎸
🧠 Technology Stack
| Component | Description |
|---|---|
| 🧩 Core AI | Google Gemini 2.5 Pro (multimodal) |
| ☁️ Cloud & ML Services | Google Cloud, Cloud Run, Vertex AI, BigQuery, Firebase, Cloud Functions |
| ⚙️ Performance Compute | Cerebras Wafer-Scale Engine |
| 🌌 Rendering & Visuals | LiquidMetal Engine |
| 💧 Cognitive Layer | Raindrop AI |
| 🎙️ Audio Layer | ElevenLabs Voice + Audio Intelligence |
| 🖼️ Inputs | Image, Video, and Text |
| 📤 Output | Structured JSON feedback |
| 💬 AI Persona | “AI-lex” — witty, expert dance judge |
| ⚙️ Processing | Client-side frame & audio extraction before secure API submission |
⚙️ Core Functionality
ChoreoRockstar features two main analysis modes, allowing users to upload still poses or short performances.
🎬 1. Keyframe Upload Mode
🧩 Analyze a single pose, key movement, or choreography concept.
👩💻 User Flow
- 📸 Upload Image
- Accepts
.png,.jpg,.webp - Max size: 4 MB
- Displays live preview
- Accepts
- 📝 Add Description
- Dance style, theme, emotion, and intent
- Dance style, theme, emotion, and intent
- 🚀 Submit for Analysis
- Sends image + text to AI-lex via Gemini API
⚙️ Backend Process
- Converts image → Base64
- Sends media + metadata → Gemini API via Cloud Run
- AI-lex evaluates:
- Posture, symmetry, expression
- Creative composition and energy
- Posture, symmetry, expression
- Returns JSON feedback → rendered via LiquidMetal UI
🎥 2. Live Performance Mode
🧩 Evaluate a 15-second video focusing on rhythm, synchronization, and energy.
👤 User Options
- 🎥 Record Live (3-sec countdown)
- 📤 Upload Video (with preview)
⚙️ Backend Process
- Frame extraction (6 evenly spaced)
- Audio isolation (PCM Base64 via Raindrop AI)
- Multimodal evaluation through Gemini + Vertex AI pipeline
- Returns structured feedback + expressive commentary via ElevenLabs synthesis
📊 Feedback & Report Generation
After analysis, users receive a neon-styled dashboard showing:
| Section | Description |
|---|---|
| 🏆 Final Score | Neon gradient display |
| 🎨 Creativity | Artistic originality |
| 🦶 Technique | Posture & precision |
| 🎵 Musicality | Beat alignment |
| 🌟 Stage Presence | Confidence factor |
| 🔮 Uniqueness | Signature style |
| 👗 Costume | Cohesion & theme |
| ⭐ Rockstar Tip | Personalized improvement insight |
Scores ≥ 85 trigger a Rockstar Splash 🎉 — confetti + sound via LiquidMetal animation engine.
🧩 UI / UX Design
| Element | Description |
|---|---|
| 🎨 Theme | Futuristic cyber-rock aesthetic |
| 🌈 Colors | Neon Cyan #00FFFF & Fuchsia #FF00FF |
| 🔠 Typography | Bold sans-serif “Rock Display” |
| 🪩 Effects | Glassmorphism, glow trails, particle confetti |
| 🎧 Sound Layer | ElevenLabs-generated voice cues |
💬 AI Persona: “AI-lex”
| Trait | Description |
|---|---|
| 🧑🎤 Role | AI Dance Judge |
| 🎭 Tone | Energetic, witty, encouraging |
| 💬 Style | Motivational + data-driven |
| ✨ Signature Line | “Keep shining, rockstar!” 🌟 |
🔒 Data Handling
- All preprocessing done client-side before submission
- Gemini + Vertex AI process only essential data (no metadata)
- Firebase handles temporary report caching
- All evaluations ephemeral — no long-term storage
🧱 Built With
🚀 Future Plans
- 🎵 Add real-time rhythm tracking with pose estimation
- 🩰 Introduce AI-generated choreography recommendations
- 🌍 Launch global leaderboard & dance community hub
- 🧠 Integrate Google Cloud Media Intelligence API for deeper performance insights
🏅 Accomplishments We’re Proud Of
- Built a full multimodal AI dance grader end-to-end.
- Integrated speech feedback via ElevenLabs and GPU inference through Cerebras Cloud.
- Designed an immersive UI with real-time scoring gauges and confetti-triggered events.
- Created AI-lex, the first AI dance judge who blends empathy with analytics.
🧩 Challenges We Overcame
- Syncing frame-by-frame analysis with beat-detection.
- Maintaining <1s response time while processing multimodal data.
- Designing natural-sounding AI voice commentary in real time.
🚀 What’s Next
- 🩰 Pose Estimation Layer — Real-time skeletal tracking.
- 🎶 ChoreoGen Mode — AI-generated personalized dance sequences.
- 🌍 Community Leaderboards — Compete with dancers globally.
- 🧭 Cross-Platform App — Native mobile version built with React Native.
🧱 Built With
☁️ Google Cloud Ecosystem
🤖 AI & ML Frameworks
💻 Web & Frontend
⚙️ Backend & Infrastructure
🧠 Developer Tools
🌟 Summary
ChoreoRockstar fuses AI, motion science, and performance art into a single dynamic web experience.
With its multimodal intelligence, voice-enabled judging, and rockstar aesthetic, it transforms dance into a measurable, gamified journey of growth.
✨ “AI-lex doesn’t miss a beat — because every move tells a story.” ✨
┌────────────────────────────────────┐
│ 👩🎤✨🎛️ User Interface (Frontend) │
│────────────────────────────────────│
│ 💻 React + Next.js (Vercel) │
│ 🎨 Neon Glassmorphism UI │
│ 🕹️ Buttons • 🎉 Confetti • 📊 Gauges │
└───────────────🔽────────────────────┘
│
▼
┌────────────────────────────────────────────┐
│ 🎥📸🧲 Media Input Layer │
│────────────────────────────────────────────│
│ 🖼️ Image / 🎬 Video Upload │
│ ⏱️ Live Record (15 sec) │
│ 🎞️ Frame Extraction (ffmpeg.js) │
│ 🎧 Audio Isolation (WASM) │
└───────────────🔽────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────┐
│ ☁️⚡🛡️ Backend API Gateway (Node.js + Express) │
│────────────────────────────────────────────────────────│
│ 🧹 Input Validation • 🧼 Data Sanitization │
│ 🧩 Preprocessing (Frames + Audio) │
│ 🔗 Routes to AI Core • 🧭 Logging & Monitoring │
└───────────────🔽────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────────┐
│ 🧠🤖🌈 AI Processing Core (Google Cloud Run) │
│────────────────────────────────────────────────────────────────────│
│ 🌌 Vertex AI (Gemini 2.5 Pro) Multimodal Engine │
│ 🖼️ Base64 Frames + 🔊 Audio → 🧠 AI Evaluation │
│ 📊 Returns JSON Feedback (Scores + Critiques + Tags) │
│ 🔄 Async Push → 🔥 Firebase • 🌊 Raindrop Streams │
└───────────────🔽────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────┐
│ 🔊🎙️🧏 Voice Feedback Engine │
│──────────────────────────────────────────────│
│ 💬 Text → 🧪 ElevenLabs API → 🎧 Speech Output │
│ 🌐 Streams via WebSocket (Real-Time Audio) │
└───────────────🔽──────────────────────────────┘
│
▼
┌────────────────────────────────────────────────┐
│ 🎨📊🎆 Visualization & Analytics Layer │
│────────────────────────────────────────────────│
│ 📈 Feedback Gauges • 📉 Trend Charts │
│ 🎉 Confetti • ✨ Rockstar Splash Animations │
│ 🗂️ Session Storage → Firebase + Raindrop │
└───────────────🔽────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────┐
│ ☁️💾🧠 Cloud Data & Compute Infrastructure │
│────────────────────────────────────────────────────────────────│
│ ☁️ Google Cloud Storage (Video Cache) │
│ ⚡ Cloud Run Auto-scaling • 🧊 LiquidMetal GPU Tuning │
│ 🌌 Vertex AI / Gemini 2.5 Pro │
│ 🤯 Cerebras Cloud (Parallel Inference) │
│ 🌊 Raindrop (Data Orchestration) • 🌐 Vultr Compute Nodes 💙 │
│ 🤖 Hugging Face / Replicate (Auxiliary Models) │
└────────────────────────────────────────────────────────────────┘
Log in or sign up for Devpost to join the conversation.