Rock Paper Scissor Game with Camera Input

This project is a classic children's game except you're playing against the computer as if it's a person! Make a hand gesture towards the camera and press play!

Comment

Inspiration

I wanted to build something that felt fun on the surface but was secretly very complex (and built with C for CMPSC 311 credit lol). Rock–Paper–Scissors is simple, fast, and instantly understandable to anyone, so it made a great testbed. But our real goal was bigger than “just an RPS game.”

So, I wanted to figure out the following questions:

  • Can I write the game logic once in C and then reuse it everywhere?
  • Can I turn that C logic into a shared library and call it from Python?
  • Can I let people play the game without touching the keyboard or mouse, using only their hand in front of a camera?

That last question is where the webcam/gesture control idea came from. I liked the idea of literally throwing hand gestures at your laptop and having it respond.

What it does

The project lets you play Rock–Paper–Scissors in three different modes, all powered by the exact same C game engine:

  1. Terminal Mode (pure C)

    • Runs in the command line.
    • You type 0 = ROCK, 1 = PAPER, 2 = SCISSORS.
    • The program prints your move, the computer’s move, and who won.

  2. Desktop GUI Mode (Tkinter)

    • You get a desktop window with buttons for ROCK / PAPER / SCISSORS.
    • There’s a “Play Again / Start Round” button that starts a visible countdown: 3…2…1…Shoot!
    • When the countdown ends, it calls the same C engine, then shows “You chose X,” “Computer chose Y,” and “Result: You WIN / Computer WINS / Tie.”

  3. Camera / Gesture Mode (OpenCV + MediaPipe) -> The most interesting one!!

    • You use your webcam instead of clicking anything.
    • It detects your hand shape in real time:
      • Fist → ROCK
      • Flat/open palm → PAPER
      • Two fingers → SCISSORS
    • During a countdown, the camera watches your hand, pick the most consistent gesture, send that to the C engine, and then display the result in the GUI.

In every mode, the “who wins?” logic is not written in Python. It always comes from the same compiled C code.

How I built it

1. Core game engine in C

With help from generative AI (ChatGPT), I wrote the core RPS logic in rps.c / rps.h. That includes:

  • Seeding the random number generator.
  • Picking the computer’s move.
  • Comparing the player move vs. the computer move to decide who wins.
  • Filling a struct rps_round_t with:
    • player_move
    • cpu_move
    • result (tie, player win, or computer win)

Also, I built a small CLI (main.c) around that engine so you can run the game directly in C with no Python involved.

2. Turning C into a shared library

I compiled the C engine into a dynamic library (librps.dylib on macOS).
That gives a reusable, compiled “game service” that other languages can call.

In Python, I load that library with ctypes.CDLL("librps.dylib"), define a matching struct in Python that mirrors the C struct, and then call the C function rps_play_round(...) to play a round.

3. GUI layer in Python (Tkinter)

Also, a Tkinter GUI (ui.py) is created so that it:

  • Renders buttons and labels.
  • Runs a countdown timer using Tkinter’s after() (so the app doesn’t freeze).
  • Lets the user pick ROCK / PAPER / SCISSORS.
  • Calls into the C library to resolve the round.
  • Updates the GUI with the results.

4. Camera / gesture layer (OpenCV + MediaPipe)

Lastly, the GUI is extended into a hybrid camera-based game (new_rps_with_cam_and_gui.py):

  • Grab webcam frames via OpenCV.
  • Run MediaPipe Hands on each frame to get 21 hand landmarks.
  • Iinfer whether the hand looks like ROCK, PAPER, or SCISSORS by looking at finger extension patterns.
  • During a 3-second countdown, keep sampling the user's hand.
  • At “Shoot!”, choose the most frequently detected gesture in that window.
  • Pass that move to the same C engine and show the final result in the GUI.

Challenges I ran into

Talking between C and Python

C uses raw structs and pointers while Python does not.
To bridge them, I had to:

  • Mirror the exact struct layout from C using ctypes.Structure in Python.
  • Set the correct argument types (argtypes) and return types (restype) for each function I call.
  • Make sure both sides agree on memory size and ordering.
    If that doesn’t match exactly, you just get garbage.

Building a .dylib and loading it

Compiling a normal C program is easy.
Compiling a dynamic library that can be safely loaded from Python is harder because of:

  • Correct compiler flags,
  • Correct architecture,
  • Making sure the .dylib ends up where ctypes expects it.

It took a lot of time and prompting to get a stable shared library that loads cleanly on macOS.

Real-time camera + GUI timing

Tkinter runs an event loop. OpenCV also wants to run continuously to read frames. A naive while True: loop would freeze the GUI. So I had to:

  • Use Tkinter’s .after() scheduling to repeatedly grab and display new camera frames,
  • Update the countdown timer,
  • Capture the “current gesture,”
  • And still keep the window responsive so buttons work.

Getting all those loops to cooperate without freezing was a real engineering problem.

Hand gesture detection

MediaPipe gives 3D landmark coordinates for the hand, not labels like “This is ROCK.”
I had to invent my own rules:

  • Closed fist → ROCK
  • Fully open palm → PAPER
  • Two extended fingers → SCISSORS
    Lighting, camera angle, and partial hands can confuse this. Tuning it to be “good enough to demo live” was surprisingly non-trivial.

Accomplishments that I'm proud of

  • With help from ChatGPT and inspiration from GeeksForGeeks, I built a C engine that is actually reusable.
    The exact same compiled code powers:

    • a terminal game,
    • a button-based desktop game,
    • and a webcam/gesture-based game.

  • I got live computer vision working in a GUI.
    It’s not just printing frames to a random OpenCV window — it’s embedded into our Tkinter UI, synchronized with a countdown, and hooked directly into our game logic.

  • I made a game where you can play with your bare hands in front of your laptop. No keyboard. No controller. Just ✊, ✋, and ✌️.

What I learned

  • How to expose C functions to Python using ctypes, including how to share structs safely across languages.
  • How to compile a .dylib (shared library) and treat C code like a service that higher-level code can call.
  • How event loops work in desktop GUI apps, and how to keep them responsive even while doing real-time video capture.
  • How to combine computer vision (MediaPipe + OpenCV) with a traditional GUI toolkit (Tkinter) without everything crashing.
  • How to think about “systems programming” as more than just low-level code. It’s also about clean interfaces, portability, and reusability across layers.

What's next for this project!

  • Better gesture recognition:
    Replace the simple “count which fingers are up” heuristic with a tiny classifier trained on ROCK / PAPER / SCISSORS hand poses. That would make it more robust to lighting, camera angle, and left vs right hand.

  • Scoreboard and stats:
    Track total wins / losses / ties over multiple rounds and display a live scoreboard in the GUI.

  • Online multiplayer:
    Let two people play each other remotely by sending just their moves over the network, but still use the same C engine to judge each round.

  • Cross-platform builds:
    Right now I build a .dylib for macOS. I want to ship:

    • .so for Linux
    • .dll for Windows
      and have Python load the correct one automatically, so anyone can run the camera-controlled RPS game on their machine.

  • Packaging / installer:
    Bundle the game so someone can just download the app and start waving their hand at their webcam instead of setting up Python manually.

Built With

Share this project:

Updates