What it does

Hackwes Basketball Gamification turns basketball practice into a fully interactive and competitive experience. Using AI models, it detects the ball, hoop, and players in real time, tracks shot trajectories, and determines whether an attempt is a hit or a miss. Players can practice live with their camera feed or upload pre-recorded footage for analysis, while the system delivers professional-grade statistics and gamified features like leaderboards, achievements, and animated scoreboards.

How we built it

We built the platform by training a custom YOLOv8 model on Roboflow datasets for hoop, ball, and player detection, using Runpod’s GPU infrastructure for faster training. The backend is powered by FastAPI and Uvicorn, providing both REST endpoints and WebSocket support for real-time analysis. OpenCV handles video frame processing while PyTorch runs the detection model and trajectory physics checks. On the frontend, we used Next.js 15 with a mobile-first, gaming-inspired interface that shows live stats, dual video feeds, and Tekken-style leaderboards, creating an engaging user experience.

Challenges we ran into

The biggest challenges were managing datasets and ensuring model accuracy across varied environments, avoiding false positives when the ball brushed past the rim instead of going through, and balancing performance so that live streaming remained smooth. Setting up training pipelines on cloud GPUs was also non-trivial, especially with large datasets and formatting requirements. On the frontend, syncing real-time data from the backend while maintaining smooth animations and responsive design presented its own difficulties.

Accomplishments that we’re proud of

We are proud of building an end-to-end pipeline that goes from real-time object detection to gamified analytics in one seamless experience. Our custom-trained YOLOv8 model performs strongly in detecting shots, and our trajectory physics module ensures reliable hit-or-miss detection. The frontend complements this with an immersive, gaming-style interface featuring leaderboards, achievements, and live HUD overlays, turning training into a fun and competitive challenge.

What we learned

Through this project we learned how to fine-tune pretrained models on custom sports datasets, how to combine AI detection with physics-based validation for higher accuracy, and how to serve these results in real time with FastAPI and WebSockets. We also explored gamification design principles and how important user experience is in keeping players engaged with analytics and competitive features. Finally, we gained practical experience working with GPU-powered cloud environments like Runpod for faster model development.

What’s next for Hackwes Basketball Gamification

Next, we aim to expand the dataset to improve detection in different courts, lighting conditions, and camera angles. We plan to integrate pose estimation for analyzing shooting form, build a global leaderboard for multiplayer competitions, and release a mobile app so players can simply use their phone cameras. Long term, we want to add multi-angle support for 3D shot analysis and develop AI-powered coaching drills that adapt to each player’s performance.

Built With

Share this project:

Updates