ISH: Shadow Boxing

• 

  Main Menu

• 

  Game Rules

• 

  Game Rules Part 3

• 

  Game Rules Part 2

• 

  Gameplay

Inspiration

I was inspired by the popular TikTok trend and thought about using Gemini's vision capabilities in a game setting. I wanted to see how that would work with real-time face movement and hand gesture detection in a browser-based game.

What it does

Ish is a high-energy, digital adaptation of the classic "Atchimuite Hoi" (Shadow Boxing) game.

• Gameplay: One player points (Attacker), and the other looks (Defender).
  • If the Defender looks in the same direction the Attacker points, the Attacker wins the point.
  • If the Defender looks away, they survive.
• AI Vision: It uses the webcam to detect Hand Gestures (Up, Down, Left, Right) and Head Orientation in real-time.
• Game Modes:
  • Solo Arcade: Play against Gemini.
  • Multiplayer: Create a room and challenge a friend on a different device (supports local network/mock testing and Firebase).

• GenAI Features:

  • Avatar Creation: Generates custom esports-style avatars based on your name using Gemini 3 ## How we built it Ish was built using **Gemini 3.0 Pro Preview API using the Google AI Studio App Builder, which made the creation of apps simple and easy. All this required was writing a few prompts to scaffold the logic.

• Frontend: React, Tailwind CSS for the stylized "Sports/Cyberpunk" UI.

• Computer Vision: MediaPipe Tasks Vision for high-performance, client-side face and hand tracking.

• Backend: A dual-mode backend service that works with Firebase Firestore (for production) or falls back to a clever LocalStorage simulation for instant testing.

Challenges we ran into

• Syncing Vision & Gameplay: Getting the "beat" right—ensuring the camera captures the player's move exactly when the countdown hits zero—required fine-tuning the animation loops and state management.
• Multiplayer Latency: Creating a smooth experience where two players verify moves against each other required careful state synchronization logic.
• Browser Permissions: Handling camera permissions gracefully across different browsers and devices.

Accomplishments that we're proud of

• Multiplayer works perfectly: The synchronization between host and guest feels snappy.
• Detection Accuracy: The face and hand movement detection is incredibly responsive.
• AI Integration: The generated avatars and commentary add a layer of polish that makes the web app feel like a premium game.

What we learned

It seems like Gemini 3.0 Pro Preview can build almost anything. When it comes to games, it knocks it out of the park, especially in generating complex logic and creative assets simultaneously.

What's next for ISH: Shadow Boxing

• Hooking this up to actual Firestore and Firebase Authentication to keep persistent data.
• Global Leaderboards to track the best players worldwide.

Built With

• aistudio
• gemini3