Inspiration
Our inspiration came from the idea of making human-computer interaction more natural and intuitive, almost like magic. We named our project "Wukong.ai" after Sun Wukong, the Monkey King, who is a symbol of playful power and transformation. We wanted to transform a simple, classic game like Tic-Tac-Toe into a new experience, moving beyond the keyboard and mouse to control it with something as simple as a hand gesture.
What it does
Wukong.ai is a multi-part project that reimagines the classic game of Tic-Tac-Toe. It features:
A core Tic-Tac-Toe game and a launcher built with the C programming language, demonstrating efficient, low-level implementation. The main feature: A second version of the game that uses Python, OpenCV, and MediaPipe to allow a player to make moves using hand gestures. The application uses a standard webcam to track the player's hand and lets them select a square on the board by simply pointing at it.
How we built it
We built the project in two main parts. The foundational Tic-Tac-Toe game and its launcher were written in C and compiled using a Makefile for a lightweight and fast experience.
The gesture-controlled version was built using Python. We used the OpenCV library to capture the live video feed from a webcam. We then fed this video into Google's MediaPipe framework, which we used to detect hand landmarks in real-time. We wrote custom logic to interpret these landmarks, identifying when a player is pointing and which square on the Tic-Tac-Toe grid they are selecting.
Challenges we ran into
Our biggest challenge was in the gesture recognition logic. It was difficult to accurately map the 3D coordinates of a hand in physical space to the 2D grid of the game board on the screen. We also had to fine-tune the system to correctly distinguish between an intentional "select" gesture (like a pointing finger) and other unintentional hand movements, all while ensuring the video processing remained smooth and didn't lag.
Accomplishments that we're proud of
We are incredibly proud of creating a working, real-time, gesture-based game interface. Seeing the game respond to a simple hand movement for the first time was a huge success. We are also proud of successfully combining two very different technologies—low-level C programming and high-level Python scripting for computer vision—into a single, cohesive project concept.
What we learned
This project was a deep dive into the world of computer vision and human-computer interaction. We learned how to use powerful libraries like OpenCV and MediaPipe to build practical applications. We also gained a much better appreciation for the complexities of creating intuitive user interfaces that go beyond traditional inputs. It was a great lesson in bridging the gap between different programming languages and paradigms.
What's next for Wukong.ai
The future for Wukong.ai is to expand beyond just one game. We want to build a framework that could allow users to play other simple games (like Pong or Connect Four) using gestures. Ultimately, we envision Wukong.ai evolving into a lightweight accessibility tool, enabling users to perform basic computer tasks and control applications using only a webcam and hand movements.
Log in or sign up for Devpost to join the conversation.