Inspiration
Anatomy is one of the most visual subjects in education, yet students are expected to learn it from flat, static textbook diagrams. Translating 2D images into 3D mental models is difficult, and memorization often replaces true understanding.
I wanted to bridge that gap.
The idea behind SkeleVision came from a simple thought: instead of imagining anatomy in 3D, what if students could see it directly on their own body in real time? By combining AI and computer vision, I aimed to transform anatomy learning from passive memorization into active interaction.
What it does
SkeleVision is an AI-powered interactive anatomy learning application that overlays a real-time digital skeleton onto the user’s body using a webcam.
Users stand in front of the camera and point at a body part with their index finger. The system detects the gesture and instantly displays:
The bone name
Scientific terminology
Anatomical facts
AI-generated explanations
Instead of flipping through pages, students physically engage with their own body to learn anatomy in an intuitive and memorable way.
How I built it
SkeleVision is built as a full-stack AI application combining computer vision, backend logic, and 3D visualization.
MediaPipe Holistic performs real-time pose detection and tracks body landmarks.
OpenCV processes live webcam video frames.
A Flask (Python) backend handles pose-to-bone mapping logic and API integration.
Three.js renders 3D anatomical visualization in the browser.
Google Gemini AI generates contextual explanations when users ask deeper questions.
Challenges I ran into
Real-time precision: Maintaining accurate landmark detection under different lighting conditions and body angles required careful tuning.
Gesture sensitivity: If the detection threshold was too large, it caused false triggers. If too small, the app felt unresponsive. Finding the right balance was critical.
Performance optimization: Running pose detection, video processing, 3D rendering, and AI responses simultaneously required efficient architecture to maintain smooth performance.
Accomplishments that I'm proud of
Successfully integrating real-time pose detection with interactive learning
Achieving responsive finger-to-bone interaction
Combining AI explanations with computer vision in a seamless experience
Building a functional full-stack AI educational tool from scratch
Demonstrating how AI can solve a real educational problem
What I learned
Through building SkeleVision, I learned:
How real-time computer vision systems process and interpret body landmarks
How to integrate AI APIs into interactive applications
The importance of optimizing performance in AI-driven web apps
How user experience directly impacts educational effectiveness
Most importantly, I learned that technology is most powerful when it makes learning intuitive and human-centered.
What's next for SkeleVision
Expanding to include muscles, joints, and organ systems
Adding quiz mode for interactive assessment
Improving gesture recognition with advanced hand tracking
Deploying it as a scalable cloud-based educational platform
Exploring integration with AR/VR for immersive anatomy learning
Log in or sign up for Devpost to join the conversation.