The Hackwarts Duelist

A zero-hardware wizard dueling game where players cast spells with webcam hand gestures, wand motion, and AI-powered magical opponents.

Comment

Inspiration ✨

I wanted to make wizard dueling feel physical and magical without requiring VR headsets, controllers, or custom hardware. The core idea was that if a webcam could track my hands and motion well enough, then a stick, pencil, or even my hand could become a wand. I also wanted something that would be fun to demo live, so I combined spell-casting with a Hogwarts-inspired setting and theatrical AI opponents.

What it does 🎮

The Hackwarts Duelist is a webcam-powered wizard dueling game that lets players cast spells using hand poses and wand-like motion gestures. It also includes AI opponents that react strategically and taunt back with voice playback, plus a Hogwarts-inspired first-person exploration mode where players can encounter NPCs and start duels. The whole experience is designed to work as a zero-hardware setup where the webcam acts as the main input device.

How I built it 🛠️

I built the project solo as a web app using React, TypeScript, and Vite, with Tailwind CSS and shadcn/ui for the interface. For the 3D environments and scenes, I used Three.js with React Three Fiber. For real-time tracking, I used MediaPipe Tasks Vision in the browser to detect hands and face landmarks from webcam video.

I implemented a custom useHandTracking hook that handles webcam input, hand and face detection, wand-tip estimation, motion smoothing, and wand trail capture. I also built a custom gesture recognizer for path-based spell casting (such as circles, lines, V-shapes, and zigzags) and combined it with static hand-pose detection. On the backend side, I used Supabase Edge Functions to power AI opponent responses and voice taunt generation.

Challenges I ran into ⚡

The hardest part was making real-time webcam interaction feel reliable and responsive. Hand tracking and motion trails are naturally noisy, so I had to spend a lot of time tuning smoothing and gesture thresholds to avoid accidental spell casts while still keeping the game responsive. That tradeoff between accuracy and responsiveness was one of the biggest technical challenges.

Another challenge was integrating AI and voice features into the gameplay loop without making the experience feel delayed or fragile. I also had to handle camera permissions, model loading, and browser edge cases gracefully so the game would still feel playable when something went wrong. Since I was working solo, scope management was also a challenge, especially while trying to keep the project polished.

Accomplishments that I'm proud of 🏆

I am proud that I built a working wizard dueling experience that feels interactive and magical using only a webcam. The project combines multiple input methods, including hand poses, wand motion gestures, and head movement, and ties them into a real gameplay loop with spells, cooldowns, mana, combat feedback, and battle stats.

I am also proud of how much personality made it into the experience. The AI opponent behavior and voiced taunts make the duels feel more dynamic, and the Hogwarts-inspired exploration mode gives the project more depth than a standalone duel screen.

What I learned 📚

I learned that computer vision projects are just as much about user experience design as they are about detection accuracy. Smoothing, calibration, forgiveness, and feedback are essential if the interaction is supposed to feel fun instead of frustrating. I also learned that gesture systems need to be designed around how people actually move in messy real-world conditions, not just ideal test inputs.

On the engineering side, I learned a lot about structuring a frontend game loop around asynchronous services like AI and text-to-speech, and about using edge functions to keep API keys and service integrations off the client. Working solo also reinforced how important it is to prioritize a strong core loop first and add polish incrementally.

What's next for The Hackwarts Duelist 🚀

Next, I want to expand the spell system with more spells, combos, and possibly custom gesture training so players can personalize how they cast. I also want to improve the AI opponents with different personalities and difficulty levels, and eventually support real-time multiplayer duels.

Beyond combat, I want to deepen the Hogwarts-inspired world with more rooms, quests, progression, and better accessibility and calibration options, including left-handed support and lighting-aware setup. I would also love to add leaderboards, battle replays, and shareable clips so the game becomes more replayable and social.

Built With

Share this project:

Updates