AR Carrom

Inspiration

Carrom is a fixture of South Asian living rooms, but you need the board, the powder, and everyone in the same room. We wanted that exact feeling, minus the board: pull out two phones, point them at a table, and play real carrom in the air in front of you.

We found existing takes on digital carrom built for VR, but a VR headset is expensive and something most people simply don't own, which puts the game behind a hardware wall. AR flips that: it runs on the phone already in your pocket. No headset, no special gear, just point your camera at any table. That accessibility is the whole point, so we went AR-first.

And if we were going to do it in AR, we didn't want a tap-the-screen port. We wanted you to flick the striker with your actual hand, the way you do on a real board.

What it does

AR Carrom is multiplayer augmented-reality carrom.

Two phones share one carrom board anchored in the same physical spot, you both look at the same real table and see the same board, pieces, and shots.
You shoot by flicking the striker with your bare hand in mid-air, no touchscreen. The camera tracks your hand; pinch to aim, release and flick to fire.
Full carrom rules: turns, fouls, covering the queen, re-racking, scoring to 25.
Haptics and procedural sound on every strike, rail bounce, and pocket, so a hit feels like a hit.

How we built it

Engine: Unity 6 + AR Foundation (ARKit) on iOS, with carrom as a planar PhysX simulation — circle colliders, tuned restitution and drag, pockets as triggers.
Shared space (the hard part): both phones agree on the board's position by looking at the same playing card as a shared image anchor, so the virtual board lands on the identical physical point for both players. They talk directly over Apple MultipeerConnectivity, peer-to-peer, no server, no internet.
Sync model: carrom is turn-based, so only the shooting phone runs physics. During a shot it streams every piece's position 15 times a second; when the pieces settle it sends one authoritative snapshot (positions + score + turn), so both boards always end identical.
The flick: a native Objective-C plugin runs Apple Vision hand-pose detection on each AR camera frame (21 hand joints), and we track the index fingertip and thumb pinch. The fingertip's path is projected onto the board plane and its direction + speed become the shot. On LiDAR iPhones we also sample real depth under the finger, so the game knows whether your finger is down at the board or just hovering, it won't fire by accident. Non-LiDAR phones are detected automatically and fall back to camera-only tracking.
Rules engine: written as pure C# and tested outside Unity against 20,000 randomized games plus scripted edge cases (queen coverage, striker fouls, penalties) to guarantee coin conservation and correct scoring.

Challenges we ran into

Making two phones agree on one point in space. Markerless ARKit co-localization drifted and sometimes landed ~90° off, so we added a card-based shared anchor and averaged the card pose over many frames to kill jitter.
Hand tracking is only ~20–30 Hz and can't see depth. A monocular camera can't tell how far your fingertip is, so a plain "project onto the plane" armed shots too eagerly. LiDAR depth as a height gate fixed the accidental fires on Pro phones; careful dwell/pinch logic covers the rest.
Pieces drifting off the board when the AR anchor moved, solved by keeping pieces kinematic at rest and only going dynamic during a shot.
A SIGSEGV crash from running person-segmentation occlusion and ARKit collaboration together on an older iPhone, which shaped our final mode choices.

What we learned

Turn-based games make AR multiplayer sync tractable, authority + a final snapshot beats fighting real-time determinism.
The "cheat" that sells the flick (project the fingertip onto the board plane) is good enough because physics smooths the rest — and LiDAR turns the cheat into something genuinely robust.
Testing game logic as plain code, away from the engine, caught rule bugs we'd never have found by playtesting.