Game Hand Signs

🌟 About the Project

Game Hand Signs is a real-time interactive combat game where players cast magical abilities using hand sign gestures captured through a webcam — inspired by Naruto-style hand seals. Instead of pressing buttons to attack, players perform specific hand signs and combo sequences to trigger abilities like Fireball, Water Ball, Heavy Attack, and Skeleton Wall.

I wanted to build something that feels like real magic casting, where your hands become the controller and the game reacts instantly.

💡 Inspiration

The main inspiration came from anime combat systems where power depends on skill + timing + technique, not just button mashing. I loved the idea of turning hand signs into a real gameplay mechanic, using computer vision to detect gestures and convert them into attacks with visual and sound effects.

🧠 What I Learned

This project helped me learn and practice:

Real-time computer vision pipelines
Using YOLOv8 (Ultralytics) for custom gesture detection
Designing reliable combo recognition logic
Building a game loop with Pygame
Managing game abilities with cooldowns, timing, and player states
Improving detection stability to avoid accidental triggers

🛠️ How I Built It

The project is built using:

OpenCV for webcam capture and defining player Regions of Interest (ROIs)
YOLOv8 to detect trained hand sign classes like:
snake, horse, hare, ram, dragon, dog
Pygame for the game loop, rendering, combat interactions, and sound playback

Each player performs signs inside their own ROI, and once a valid combo is recognized, the game triggers the corresponding ability with VFX + SFX.

✨ Ability Combos

Ability	Combo
🔥 Fireball	`snake → horse`
💧 Water Ball	`hare → ram`
💥 Heavy Attack	`dragon → dog`
🛡️ Skeleton Wall	`hare → snake`

⏱️ Cooldowns (Game Balance)

To keep the game fair and prevent ability spam, each spell has a cooldown timer.
This can be modeled as:

[ \text{CanCast} = \begin{cases} 1, & t - t_{\text{last}} \geq C \ 0, & \text{otherwise} \end{cases} ]

Where:

(t) is the current time
(t_{\text{last}}) is the last cast time
(C) is the cooldown duration (seconds)

⚔️ Challenges I Faced

1) Real-time detection stability

The biggest challenge was preventing the game from reacting to noisy detections. A single wrong prediction could break a combo or trigger the wrong ability. To solve this, I added logic to stabilize the detected sign over multiple frames before confirming it.

2) Combo timing and responsiveness

Combos must feel fast and smooth, but also fair. I had to balance:

responsiveness (instant casting)
accuracy (avoiding false triggers)
cooldown design (preventing spam)

3) Two-player interaction on one camera

Splitting the screen into two ROIs was a simple but effective way to support two players in one match. The challenge was ensuring both sides were equally detectable and didn’t interfere with each other.