PocketIQ

Tired of missing shots in pool? Take a picture of the table with PocketIQ and it will give you the best shot and how to execute it perfectly.

Comment

Inspiration

We like to play pool often, but most times we just hit the ball and hope it goes in, which most of the time it doesn't. We thought it would be cool to make a software that tells you the best shots and how to execute them. We made the app web based so that anyone can access it on any device without having to download an app or anything. We're hoping to turn more mediocre pool players into pros.

What it does

PocketIQ is a phone-based 8-ball shot coach that tracks the whole game from photos.

  • Reads the table. It finds the felt and every ball in your photo, and works out which ball is which.
  • Suggests the next shot. It draws every legal shot on the table, and tapping one shows the aim, cut angle, power, spin, and where the cue ball ends up.
  • Coaches in plain words. An AI coach picks a shot and explains it using your own make rates (for example, "you make 82% of cuts under 15°"), and can read the tip aloud so both hands stay on the cue.
  • Tracks the game for you. By comparing each photo with the last, it detects pocketed balls, scratches, ball in hand, groups, whose turn it is, and the winner. Nobody types in a score.
  • Shows a top-down table you can correct. Nothing is saved until you confirm. Retake as often as you like, or fix the layout by hand (relabel, move or remove balls, undo/redo).

How we built it

Frontend: React 19, TypeScript, Vite, and Tailwind CSS, built mobile-first. A canvas draws the table in table units, and a client-side shot finder draws shots instantly.

Backend: FastAPI and Python.

  • Vision (OpenCV): a LAB chroma mask finds the felt, four fitted edges plus the side-pocket gaps orient the table, and a homography warps every photo into a shared 1000 × 500 table space. Balls come from either a Hough circle search or a YOLOv8n model we trained and exported to ONNX. Each ball's number is classified from color features, then refined with swaps so no number is used twice. Gemini can optionally give a second opinion on ball numbers.
  • Game engine: pure functions on a plain dict. Vision only proposes detections; the game memory decides what they mean. It reconciles detections with what is known to be in play, applies the 8-ball rules, and can rewind and replay the latest photo when you correct it.
  • Physics: ghost-ball geometry finds direct and bank shots. A friction simulator (sliding then rolling, ball and cushion collisions, draw, follow and english) then grid-searches power and spin and keeps the combination that pockets the ball and leaves the easiest next shot.
  • Coach: Gemini never does geometry. It receives the candidate shots, each with a physics plan, plus the player's real stats, and returns one pick with short tips as structured JSON. ElevenLabs reads the advice aloud.
  • Data: Tiger Data (Postgres + TimescaleDB) stores each game as JSONB and every shot in a hypertable, which feeds per-player make rates by cut angle, shot kind and pocket.

Challenges we ran into

  • Photos are messy. Lighting, glare, and different cloth colors make it hard to find the felt and tell a striped ball from a solid one. We moved from circle detection to a trained YOLO detector and kept the circle search as a fallback.
  • Telling balls apart. Classifying each ball in isolation produced duplicate numbers, so we assign all balls at once and use swaps to ensure no number repeats.
  • Detections are never perfect. A hidden ball looks like a pocketed ball, and a missed detection looks like a scratch. We made the game state authoritative and built reconcile, revive, and replay logic so any mistake can be fixed without breaking the game.
  • Keeping the LLM honest. Language models are bad at geometry, so we split the work: code computes every shot, and Gemini only chooses among candidates and explains them.
  • Realistic but fast physics. We modeled friction, spin, and cushion rebounds well enough for useful cue-ball positioning, but squirt, swerve, and throw are not modeled yet.

Accomplishments that we're proud of

  • A shot planner that picks power and spin to set up the next shot, not just to pocket the ball.
  • A trained on-device-style ball detector, a rules engine, a physics simulator, and an AI coach working together.
  • A full game tracked from photos alone, with no manual scorekeeping.
  • A preview-then-confirm flow, so a bad photo never corrupts the game.
  • A coach that grounds its advice in your own make rates, not generic tips.

What we learned

  • Let code do the math and let the LLM do the words. Constraining Gemini to pick from computed candidates made its advice far more reliable.
  • Treat vision as a suggestion, not a fact. Making game memory authoritative and every photo correctable made the whole app more trustworthy.
  • Build for the person holding the phone: free retakes, one-tap fixes, and a landscape mode all came from thinking about someone mid-game with a cue in one hand.

What's next for PocketIQ

  • Support for 9-ball, straight pool, and snooker.
  • Better physics: squirt, swerve, and throw.
  • A larger training set for more robust ball detection in poor lighting.
  • Practice drills, long-term progress charts, and shareable match recaps.

Built With

Share this project:

Updates

Submission history