Pickleball: On The Go

Inspiration

We were inspired by the rise in Pickleball interest and also the lack of pickleball courts, so we made this game! You can practice your pickleball on the go, when you're waiting at the airport, at home, or even waiting for a physical Pickleball court.

What it does

Your phone is the paddle; if you swing your phone, your virtual pickleball paddle also swings. The program also uses Computer Vision to track your position on the court; if you need to move left, you have to physically move left and hit the ball.

How we built it

We built the game with JavaScript, Three.js, WebSockets, and Node.js. The phone streams motion data from its accelerometer and gyroscope, while the laptop uses MediaPipe Pose Landmarker for body tracking. A server-authoritative simulation manages the ball, game state, scoring, and player positions. NVIDIA Nemotron models classify swings and review rally events for rule violations, with deterministic fallbacks when the model is unavailable.

Challenges we ran into

The biggest challenges were making phone motion data reliable, reducing camera-tracking jitter, and synchronizing multiple devices over a local network. We also had to design generous hit detection so the game felt responsive instead of frustrating. IOS motion permissions and HTTPS requirements added another layer of setup, while integrating AI models required careful validation so an incorrect model response could never change the score or game state. We created a playable physical pickleball experience without requiring a ball, paddle, or court. Players can swing their phones, move around in real space, and see those actions reflected immediately in the virtual game.

Accomplishments that we're proud of

We created a playable physical pickleball experience without requiring a ball, paddle, or court. Players can swing their phones, move around in real space, and see those actions reflected immediately in the virtual game.

What we learned

We learned that responsive game design is often more important than perfect physical realism. Small delays, jitter, or overly strict hit detection can make a game feel broken, so generous timing windows and clear feedback are essential.

What's next for Pickleball: On The Go

Next, we want to improve pose tracking, support more devices, add multiplayer matchmaking, and expand the game beyond singles. We would also like to add player progression, training drills, shot analytics, customizable courts, and more accurate swing classification. Long term, our goal is to make this game a way to practice movement

Built With

• javascript
• nemotron
• pickleball
• websocket