SRIKA

System of realtime intent based kinamatic analysis

Comment

Inspiration

We wanted to close the gap between human physical movement and competitive digital gameplay.

Modern gaming is incredibly immersive, yet interaction remains largely static.
Players sit for long hours, often leading to posture issues, reduced activity levels, and long-term health concerns.

At the same time, advanced motion experiences typically require expensive hardware such as VR headsets or specialized sensors, putting them out of reach for many players.

Our goal with Srika was to make body movement feel like a native controller using only a webcam — while also providing real-time understanding of why a move succeeds or fails.

What if anyone, anywhere, could step into a more physical, intelligent way to play?

What it does

Srika converts a standard webcam into a real-time game controller.

The system captures body landmarks, evaluates them using an anatomical rule engine, and injects validated inputs directly into games like Tekken 8.
When a move fails validation, Srika calls Gemini to transform low-level biomechanical rejection signals into short, human-readable guidance.

This enables:

  • more active gameplay
  • awareness of movement quality
  • deeper immersion
  • controller-free interaction without expensive equipment

The result is a hybrid model:

  • deterministic speed from rules
  • adaptive intelligence from AI

How we built it

Real-Time Input Control Loop (<30ms)

Pose landmarks are captured in the frontend and streamed into an Electron-isolated Python bridge.
Inside the bridge, a preset-specific Rule Engine evaluates vectors, joint angles, and anatomical gates.
Only validated actions are converted into virtual controller events.

This ensures responsiveness fast enough for competitive play.

Multi-Window Electron Architecture

We run a control dashboard and a live HUD.
State is synchronized through IPC, allowing instant feedback while the player remains immersed in the game.

Gemini as an Advisory Layer

To avoid adding delay, Gemini operates outside the control loop.

When a rejection occurs:

  1. immediate rule-based feedback appears
  2. an asynchronous Gemini request is triggered
  3. natural language guidance replaces the fallback

This gives players intelligence without sacrificing speed.

Accessible by Design

Unlike VR or specialized hardware solutions, Srika works with existing consumer devices.
A webcam becomes the gateway to physical, interactive gaming.

Challenges we ran into

Balancing speed with intelligence.
AI responses require time, so we created an “Instant Feedback → AI Enrichment” model.

Maintaining system isolation.
Hackathon and production environments had to coexist safely.

Display priority management.
Games often dominate the rendering pipeline, requiring careful overlay handling.

Accomplishments that we're proud of

  • Delivering physical interaction without extra hardware
  • Preserving competitive latency while integrating AI
  • Separating deterministic execution from generative interpretation
  • Demonstrating a path toward healthier, more engaging gameplay

What we learned

AI is most powerful in interactive environments when it augments human action rather than replaces it.

Rules ensure reliability.
AI provides meaning.

Together they create a new relationship between player and system.

What's next for Srika

We want to expand beyond fixed mappings into adaptive motion intelligence.

Future directions include:

  • generating interaction models from expert demonstrations
  • cloud-based preset ecosystems
  • multiplayer and social play
  • applications in fitness, rehabilitation, and accessibility

Our long-term vision is to make natural human movement a universal interface to digital worlds.

Built With

Share this project:

Updates