Inspiration

As a kid, I used to walk two fingers across the kitchen table kicking gum balls and imagining my fingers as tiny people running, jumping, kicking and fighting. With mixed reality and hand tracking, I finally had the tools to bring that childhood daydream to life, turning a real table into an interactive toy box.

What it does

Finger Soccer turns any flat table into a miniature mixed reality soccer field. A small pitch floats in front of you. Pinch its handle, set it down on your table, and the whistle blows as the ball drops onto the centre spot. Move your hand over the pitch, and your index and middle fingers become a tiny soccer player wearing little soccer shoes and knee-high socks. You can dribble with gentle touches, flick to kick, and score goals using only your hands. A goal brings a whistle, confetti from the posts, a scoreboard update and a roaring crowd, and the crowd groans at near misses. Both hands can play at once, you can pinch the edge of the pitch to move or turn it, and the game remembers where you left the field the next time you play in the same room.

How we built it

I built the game in Unity with the Meta XR Interaction SDK and passthrough. I created a simple miniature pitch and goal, adapted a pair of soccer shoes from the Asset Store, and modeled custom knee-high socks that fit right onto the tracked finger joints. There's no room scan: you set the field down on the table yourself, and spatial anchors remember that spot for next time. Finger movements become physics-based kicks and dribbles. Every sound is placed where it happens in the room, from ball bounces off the walls and posts to the stadium crowd.

Challenges we ran into

I've built many hand-tracking projects before, including a full thumb-based joystick. Even so, tracking fingers pointing down at a table was much harder than I expected. In a "walking fingers" pose, the back of your hand hides the fingers from the headset's cameras, and the hidden finger slowly drifts and curls. To fix it, I rebuild the hidden finger from its knuckle position and pin standing fingers to the table so they don't float. I also wrote my own kick detection: a swept collision test between physics steps, so fast flicks don't pass through the tiny ball, and a smoothed fingertip velocity, so tracking jitter doesn't cause random kicks.

Accomplishments that we're proud of

Bringing a childhood daydream to life. Seeing tiny soccer shoes and socks snap onto your fingers the moment your hand comes over the pitch brings an instant smile, with almost no instructions needed.

Making kicks feel responsive despite the tricky downward hand angle. Getting reliable, satisfying ball contact from fingers the headset can barely see took many rounds of testing, and it was worth it.

What we learned

Designing for downward-facing tabletop poses is very different from standard forward-facing hand interactions. Because your hand naturally blocks the cameras when pointing at a table, the game can't rely on raw tracking data alone. It has to repair, smooth and forgive, so the experience feels right even when the tracking doesn't.

What's next for Finger Soccer

  • PvP
  • 60-second blitz matches
  • Corner kicks and other set pieces
  • A goalkeeper (the model is already in progress)
  • NPC teammates and opponents
  • Multiplayer: head-to-head matches online or in co-location
  • More finger-dude games: fighting, running and other tabletop sports with the same characters
  • Better outfits: more convincing kits, maybe even shorts

Built With

Share this project:

Updates

Submission history