Jiggy

Home Page UI
Challenge Mode UI
Computer Vision Tracking 1
Computer Vision Tracking 2

Inspiration

With apps like TikTok making short-form dance content mainstream, we thought—why not turn dancing into a game where you can challenge friends or practice and get real-time feedback on how closely your moves match?

What it does

Jiggy lets users upload a dance video (from TikTok or elsewhere) and either challenge friends or enter practice mode. Using computer vision, it compares body movements between the source video and the user's live or uploaded video. Accuracy is scored in real-time using color-coded feedback:

🟢 Great match 🟡 Okay match 🔴 Needs improvement

How we built it

Pose Detection: Used MediaPipe to extract joint coordinates from both uploaded and live webcam videos. Angle Calculation: Used NumPy and trigonometry to calculate joint angles. Data Handling: Stored and loaded pose data using Pickle (.pkl) files and Pandas. Real-time Comparison: Leveraged WebRTC for webcam feed, and Socket.io for syncing pose data in real time.

Web App: Built the frontend with HTML/CSS, and backend using Python (Flask).

Challenges we ran into

Pose Comparison Logic: Initially tried comparing joint positions in 5D graphs which was complex. Switched to moving average of joint angles for better results.

WebSocket Instability: Encountered issues configuring URLs to public state and managing multiple users on the same WebSocket.

Client-side Rendering: Transferring uploaded video to client side via WebSocket caused unexpected bugs.

Live Feed Sync: Connecting Python script to Node webcam feed using Flask was unreliable and led to inconsistent launches.

Accomplishments that we're proud of

Created a working proof-of-concept that shows real-time dance pose comparison. Developed a color-coded accuracy scoring system that provides intuitive feedback. Successfully integrated MediaPipe, WebRTC, and Flask to create a unified app experience.

What we learned

Handling live media streams and syncing pose data across two video sources is harder than it sounds. Importance of keeping things modular—small changes in one component (like socket routing) can break the whole chain. Real-time feedback systems require performance-optimized pose comparison techniques

What's next for Jiggy

Improve scoring logic using cosine similarity and temporal smoothing. Add leaderboards, streaks, and badges to boost engagement. Optimize for mobile and support AR pose overlays for visual alignment cues.