Polaris

CV Powered Open World Collaborative Fitness Game. Run Together. Save the Arctic.

Comment 1

Inspiration

The pandemic showed us that home fitness needs a social element to stay engaging. While fitness apps track solo workouts, they miss the energy of exercising with friends. We wanted to create a collaborative fitness experience that captures the motivation of group workouts while staying accessible from home. When we discovered we could use computer vision to track multiple players from a single camera, we realized we could turn any living room into a shared fitness arena. The polar bear theme emerged naturally—they work together to survive, just like friends pushing each other to stay active.

What it does

POLARIS transforms your body into a game controller through real-time computer vision. Players physically run, jump, and climb to control polar bear characters racing through dynamic 3D worlds. Our software intelligently parses a single camera feed to track two players simultaneously, enabling side-by-side competition with sub-300ms latency. Real-time fitness metrics display calories burned, heart rate, and distance traveled, while procedurally generated tracks ensure every race feels fresh. The bears collect resources and rescue Arctic friends, turning a workout into an adventure.

How we built it

We engineered a complex technical stack from the ground up:

  • Custom CV Pipeline: Built a dual-player tracking system that parses single camera feeds with custom ROI detection and predictive interpolation
  • Low-Latency 3D Engine: Developed a custom Three.js renderer with optimized draw calls, achieving 60+ FPS with complex shaders and particle systems
  • Open World Architecture: Implemented infinite world generation with seamless chunk loading and dynamic LOD systems
  • Original Pixel Art: Created frame-by-frame animations for all character movements and environmental assets
  • Real-time Synchronization: WebSocket architecture with custom binary protocols achieving <16ms frame times
  • ML Movement Pipeline: Engineered pose-to-action mapping using sliding window analysis and gesture confidence scoring

Every component was built for performance—from the custom video stream splitter to the optimized collision detection system.

Challenges we ran into

Building a truly open-world 3D game in 36 hours required solving multiple complex problems simultaneously. Our custom CV pipeline needed to handle occlusion, varying lighting, and player crossover while maintaining real-time performance. The open-world system demanded innovative memory management—we implemented object pooling, frustum culling, and dynamic asset loading to prevent frame drops. Creating smooth, responsive controls from noisy pose data required developing a custom filtering system with predictive motion smoothing. The biggest challenge was maintaining sub-300ms end-to-end latency while processing dual video streams, running ML inference, and rendering complex 3D scenes.

Accomplishments that we're proud of

We built a full 3D open-world game with a custom engine in 36 hours—something most teams wouldn't attempt in a game jam, let alone with CV controls. Our dual-player single-camera solution is genuinely innovative, achieving latency comparable to traditional controllers. The custom pixel art and smooth animations rival commercial indie games. We're especially proud of our ML pipeline that converts human movement into fluid character control with zero configuration.

What we learned

Building a custom game engine taught us that performance optimization is about making hard architectural choices early. We discovered that computer vision at 60 FPS requires thinking in microseconds, not milliseconds. Creating engaging gameplay showed us that technical excellence means nothing without polish—those extra hours on particle effects and sound design transformed a tech demo into an experience. Most importantly, we learned that pushing technical boundaries while maintaining playability requires constant iteration and ruthless prioritization.

What's next for Polaris

POLARIS has the technical foundation to become a platform, not just a game:

  • Multiplayer Evolution: 4+ player support with advanced pose disambiguation
  • Engine Licensing: Our CV-to-game pipeline as middleware for fitness developers
  • World Creator: Node-based editor for community-designed fitness challenges
  • Advanced Biometrics: Heart rate variability and form analysis for adaptive difficulty
  • Enterprise Platform: White-label solution for gyms and wellness programs

The bears are just the beginning. We've built the technology to make any movement-based game possible.

Built With

Share this project:

Updates