PaddleLab XR

Practice your ralies
Get coached through detailed scenarios
Play anywhere- no partner needed

Inspiration

Table tennis is fast, technical, and precise, but consistent practice typically demands dedicated space, specialized tables, and available partners. For players looking to drill fundamentals or get reps in outside standard gym hours, finding physical facilities simply is not always practical.

We built PaddleLab to bring high-fidelity practice into any room. By pairing computer vision tracking for physical paddles with immersive VR controls, the platform bridges real-world muscle memory and virtual training. Whether holding an actual paddle in front of a webcam or swinging a VR controller, players can run drills, refine their mechanics, and compete anytime without needing a full-sized physical table.

What it does

PaddleLab is a responsive table tennis training environment and competitive platform built for immersive control schemes:

Dual-Mode Input Tracking: Supports both computer-vision-tracked physical paddles and fully immersive VR controllers, letting users train with natural hand-eye coordination.
Targeted Drills and Rally Sessions: Guided practice routines designed to improve reaction timing, paddle angles, and return consistency against customizable virtual drills.
Head-to-Head Multiplayer: Real-time online matchmaking and custom lobbies where players can rally, scrimmage against friends, or compete across supported setups.

Tracks

Each sponsor has a role in the product, turning technical integrations into clear, user-facing capabilities:

[Sponsor] Tiger Data — Leaderboard and Telemetry: Serves as the durable data plane via Tiger Cloud (TimescaleDB). leaderboard_entries tracks canonical per-category rankings, while a coaching_events hypertable logs stroke scores and match results for real-time telemetry and deep post-match analytics.

[Sponsor] OpenAI — Per-Shot Coaching: Transforms raw local stroke telemetry into immediate, actionable feedback. The OpenAI Responses API analyzes real-time performance numbers to generate one precise technique correction per shot, instantly displayed in the coaching panel and voiced aloud.

[Sponsor] Baseten — Post-Match Analysis: Powers comprehensive performance breakdowns using hosted zai-org/GLM-5.3-Fast inference on Baseten's OpenAI-compatible endpoint, delivering deep, fast post-game feedback with automated Gemini fallback for high reliability.

[Sponsor] Gemini — Holistic Match Summary: Generates big-picture post-match summaries by evaluating overall game flow, strategy, and performance patterns. Acts as both the primary holistic narrator and a seamless failover during high-traffic inference spikes.

[Sponsor] Backboard — Player Memory and Recurring Trends: Endows the AI coach with long-term memory across sessions. By appending stroke scores and match results to persistent Backboard threads, it analyzes player history to highlight recurring tendencies—like timing that runs early or an open paddle face on pushes.

[Sponsor] ElevenLabs — Dynamic Narration: Transforms purely visual UI text into an immersive audio experience. Uses two distinct narrator voices to dynamically read per-shot corrections, match summaries, and recurring trends aloud, with persistent user voice preferences.

[Sponsor] Linq — iMessage Invitations: Streamlines multiplayer onboarding via SMS/iMessage. Players trigger custom room invites directly to a contact's phone number through Linq's Partner API, allowing friends to jump straight into a match from their text thread without manually copying URLs.

[Sponsor] Devin — Planning and End-to-End Testing: Served as an AI teammate throughout the build, driving product architecture planning, automated end-to-end testing against production deployments, and continuous developer assistance.

How we built it

Built the core table tennis environment in VR, configuring the spatial scale, paddle collision bounds, and low-latency interaction loops.
Developed a computer vision pipeline using a webcam feed to track physical table tennis paddle orientation and positional coordinates in real time.
Implemented realistic ball physics, tuning bounce restitution, trajectory arcs, and spin dynamics to faithfully mimic table contact and air drag.
Created a cross-input multiplayer network layer that syncs paddle transforms and ball states between players on VR headsets and players using vision-tracked paddles.

Challenges we ran into

Camera Field of View & Range: Natural table tennis strokes (such as broad forehands, backhands, and deep follow-throughs) frequently carried the player's paddle outside the standard webcam's field of view. We had to implement recovery logic so the virtual paddle would not freeze or jump erratically when re-entering the frame.
CV with Ping Pong Paddle: It was hard for the webcam to detect the angles and the depth of the paddle. We decided to put ArUco markers on the paddle to solve this. However, we still experienced a bunch of random jumps in the paddle, so we smoothed it out using a smoothing filter algorithm and path interpolation during fast swings. Even after that, it was still really hard to hit the ball using the paddle, so we added an auto-aim assist to make the swings seem natural.

Accomplishments that we're proud of

Dialing in the ball physics to create a genuinely realistic VR table tennis simulator that feels natural off the face of the paddle.
Successfully implementing functional cross-input multiplayer, letting a player holding a real paddle in front of a camera rally seamlessly against a player in VR.

What we learned

First-time VR development workflows, particularly around asset optimization, rendering budgets, and keeping frame times rock solid to prevent motion sickness.
How to balance system latency against perceptual rendering speed in fast-paced, high-reflex interactive environments.
The physical ergonomics of building VR experiences around active sports, where player movement, reach volume, and room boundaries dictate gameplay design.

What's next for PaddleLab

Dedicated Hardware Accessories: Prototyping dedicated paddle adapters and clip-on controller mounts for VR to replicate the authentic weight distribution and grip of a real racket.
Wider-Angle Multi-Camera Tracking: Expanding the vision setup to support wide-angle or multi-camera configurations to capture full athletic swings without boundary clipping.
Shot Analytics & Ghost Sparring: Adding trajectory tracing and ghost-opponent replays to help players visualize shot arcs and analyze their mechanics over time.