FitCommit
Summary of the system architecture
The Home dashboard tracks your daily streak and flags unverified sessions that require immediate proof to avoid a fine
The New Appeal screen allows users to plead their case to the strict multimodal AI judge before a penalty is processed

FitCommit

Enforcing workout consistency using financial penalties and multimodal AI agents.

Problem and motivations:

Everybody has had that moment - ahh, I can't be bothered to go to the gym today, I'll go tomorrow. This is because there is no short-term pain in not going, only long-term when you haven't achieved your goals.

Living in a university environment where deadlines are constant and stress is high, we’ve all fallen into the same trap. We realized that motivation is unreliable. It flees the moment a coursework deadline appears.

We looked for apps to help make us go to the gym, but they all focused on tracking workouts—counting reps and logging miles. None of them actually solved the struggle of showing up.

So we built FitCommit. We realized we didn't need another streak tracker, but instead short-term, tangible consequences. We built an AI-powered discipline enforcer that uses financial loss to make skipping the gym more painful than actually going.

What is FitCommit?

FitCommit is an accountability platform with tangible reality checks. The concept is simple:

You commit to a workout schedule.
If you miss a session, you are automatically charged a fixed fine (which you can set).
That fine is donated to a charity of your choice.

How FitCommit verifies a workout

To check whether the user has actually gone to the gym:

The app checks location history to determine whether they were at their gym for a reasonable duration.
If this can’t be determined, the user can upload a picture taken in the gym (We use AI to verify the validity of the image ). -Just in case users might turn up to the gym then leave, we actually check how many reps they did using their camera, and an AI agent.
For runners outside the gym, Strava activity is checked to see whether they actually ran or not.

Appeal System (because life happens)

Maybe you got injured or you had to attend a family event. To solve this, we built a robust Appeal System using an AI Agent to determine whether you had a valid reason to skip your workout.

The Judge

When you submit an excuse, our multimodal AI:

Analyzes your text
Cross-references it with location history / photo metadata / Strava activity
Tracks your excuses over the previous 30 days, learning patterns to prevent you from lying

The Negotiator

If the reason is deemed valid, the agent understands and suggests an alternative solution—such as a home workout—to help you stay on track.

The Executioner

If the agent deems you to be lying or your reason to be unjustified, then a donation will be made unless you complete your workout by the end of the day.

Effort accountability inside the gym

FitCommit also helps ensure accountability within the gym using on-device computer vision to count reps, ensuring the user is putting reasonable effort into working out.

How we built FitCommit

Frontend: React Native+TypeScript (deploy to both iOS and Android from a single codebase)
Backend: Python + Django (REST framework) (robust and scalable) -Agent Kotlin (Ktor) + Python (MediaPipe) for AI/verification
Payments/Donations: Stripe (automated, reliable, secure)

Our technical architecture

The real technical innovation lies in our use of Kotlin’s Koog framework for our AI Agents. We took this hackathon as a challenge to move beyond simple LLM wrappers. We used Koog to create a true Multimodal Agent.

By building custom annotation tools in Kotlin, we gave the LLM the ability to:

“See” photos
“Read” raw GPS logs

This allowed us to build an agent that investigates and detects lies to keep you accountable. Kotlin’s abstraction allowed the agent to reason across different data types, resulting in an AI agent that can detect lies and understand the user’s behaviour.

To track the reps a user actually did using their camera, e.g 10 squats, Django sends the video URL to the Kotlin agent; the agent uses ffmpeg to get frames, a Python MediaPipe script to get knee (and torso) angles per frame, then Kotlin to count reps with a knee-angle state machine and score form, and returns rep count + status + issues/coaching so the app can show “verified” or “not accepted” with reasons. We used a similar method for gym image verification.

We also integrated the Strava API so that running users can be held accountable too—ensuring FitCommit works for both gym-goers and runners.

What challenges did we face?

Koog learning curve: We initially faced challenges in utilising Koog and creating our custom annotation tools, due to it being a new system to us. After working with the JetBrains team, we got the agent working—afterwards, replicating agents and letting them work together was quick and simple.
Prompt engineering balance: The agent was either too gullible (believing every excuse) or too strict (rejecting appeals and creating unreasonable demands). We fine-tuned prompts and tool definitions to make it strict but fair—and even understanding.
Backend ↔ agent integration: Connecting our Python/Django backend with Kotlin agents required HTTP requests between the two, which initially had latency and delays. With refinement, we reduced wait times and made it smoother for users.

What did we learn?

This hackathon was a crash course in new technologies - teaching us how to learn them fast, and how to apply them faster.

Key lessons

Complex request systems: Building a robust request system between two distinct environments—our Python/Django backend and the Kotlin Koog microservice.
Building tools for AI: Moving past simple prompt engineering and learning to build functional tools within the Koog framework. Giving agents access to APIs and real data let them verify truth rather than guess.
Feasibility & focus: We started with many ideas, but learned to debate and filter them. Instead of building a mediocre app with unrelated features, we focused on one key idea: user accountability.
API constraints matter: Understanding the limitations of our APIs was as important as the code itself. This helped us build a working product rather than a half-finished concept.

The Future

We see huge potential in changing the approach to discipline. Some potential additions include:

Expanding the appeals system
- Checking local weather data if the user claims bad weather for skipping a run
- Checking traffic data to verify user claims
Allowing users to set minimum duration or distance for runs to ensure they train hard

We believe FitCommit solves the biggest problem in personal fitness: consistency. We’re excited to keep expanding and refining it.