Inspiration

Modern fitness and performance tools generate massive amounts of data, yet most athletes and everyday users still rely on generic feedback or post-workout analysis.

The inspiration for Omnexa came from a simple question:

What if raw biometric data could be interpreted in real time with the reasoning ability of an expert coach?

We wanted to close the gap between motion capture, training history, and nutrition by building a system that understands the human body as a connected, dynamic system rather than isolated metrics.

What We Learned

Building Omnexa highlighted the importance of structured data when working with large multimodal models.

• Amplifying Reasoning: We learned how Gemini 3’s reasoning capabilities can be significantly amplified by providing clean biomechanical inputs (such as joint angles and motion phases) and by enforcing structured outputs through response schemas.
• User Experience: We also gained practical experience designing low-latency feedback loops, where inference speed and clarity of feedback directly impact the user experience.

How We Built It

Omnexa was built by combining real-time pose estimation using MediaPipe with Gemini 3 as the central reasoning engine.

1. Motion Analysis: Joint angles and movement phases are computed and passed to Gemini 3, which evaluates exercise quality and returns structured JSON feedback that drives the user interface.
2. Audio Coaching: Feedback is generated using Text-to-Speech, enabling hands-free corrections during training.
3. Session Tracking: The platform logs exercises, sets, repetitions, and total volume.
4. Nutrition Vault: Gemini 3 parses natural language meal entries and computes personalized macronutrient targets.

Concepts such as total daily energy expenditure are modeled using Gemini reasoning.

Challenges

• Real-time Feedback: One of the main challenges was achieving reliable real-time feedback without overwhelming the user. Balancing inference latency, structured outputs, and meaningful coaching cues required careful prompt design and iteration.
• Biomechanical Accuracy: Another challenge was ensuring biomechanical feedback remained accurate across different body types and movement patterns, which reinforced the need for precise kinematic representations and conservative reasoning thresholds.

Built With

• gemini
• mediapipe
• node.js
• react
• tailwind
• typescript
• vite