PocketTrainer

screen 1
screen 2
screen 3

Inspiration

We are facing a double-burden health crisis. Globally, over 1 billion people live with clinical obesity due to sedentary urban lifestyles and poor eating habits (WHO 2024). Simultaneously, physical inactivity is projected to cause 500 million new cases of preventable non-communicable diseases by 2030, costing healthcare systems USD 300 billion (WHO & Lancet 2024).

When people try to take control of their health by starting to exercise, they hit a barrier: hiring a Personal Trainer (PT) in Indonesia is a luxury that costs IDR 2,000,000 to IDR 5,000,000 per month. Without guidance, beginners turn to self-practice, leading to an alarming 27% injury rate among general gym-goers due to poor form (NIH PMC). In fact, unsupervised weightlifting and "ego lifting" fueled by social media trends have driven a 33% increase in gym-related head and face injuries (Journal of Craniofacial Surgery 2024), while regular yoga practitioners face a 10% annual injury rate from overstretching without stability (Sydney Sports Medicine / MDPI).

Existing AI fitness apps solve this by streaming camera feeds to cloud servers. This, however, drains users' internet quotas in Indonesia, introduces lag, and creates massive privacy concerns. We built PocketTrainer to democratize safe fitness: providing a real-time personal coach that runs completely offline, 100% private, and at zero server cost directly on the user's phone camera.

What it does

PocketTrainer is an on-device movement intelligence platform with a deterministic, adaptive training engine. It works through a continuous feedback loop:

Baseline Movement Assessment: During onboarding, users take a 2-minute movement scan to estimate their joint flexibility, balance, and baseline Capability Scores.
Adaptive Training Plans: Instead of static calendars, a deterministic rule engine generates personalized workout plans. These plans dynamically scale exercises up (progression) or down (regression) based on historical performance.
AI Posture Guidance: While exercising, the app uses the front-facing camera to track 33 body landmarks in real-time. It provides instant audio and visual cues (e.g., "Lower your hips slightly" or "Keep your chest upright") to correct posture and prevent joint strain.
Healthy Diet Plan: An on-device food image scanner allows users to photograph meals, which parses details using localized OCR food databases, letting users confirm and edit calorie/nutrient estimates.
Gamified System: Users earn XP, maintain streaks, and advance their "Movement Passport" to unlock higher fitness tiers. The gamified XP ledger is validated by physical movement tracking, keeping leaderboards cheat-proof.

How we built it

PocketTrainer is built using a high-performance cross-platform and native hybrid architecture:

Frontend UI: React Native with TypeScript for a polished, responsive user interface and seamless cross-platform navigation.
Native Edge-AI Engine: Custom C++ wrappers and Native Turbo Modules (Kotlin/Java on Android, Swift on iOS) wrapping MediaPipe Pose Landmarker to perform real-time spatial calculations.
Camera Frame Isolation: To ensure smooth performance, raw camera frames are captured, processed, and evaluated entirely in native memory, and immediately discarded. Only lightweight coordinates and coaching events are sent to the JavaScript layer.
Explainable AI (XAI) Rule Engine: Rather than utilizing unpredictable deep learning black-box models, coordinates are evaluated geometrically (acting as a digital goniometer) using deterministic, physiotherapist-vetted state machine rules.
Local Caching & Backend Sync: Encrypted SQLite for offline database operations, synchronizing with a lightweight NestJS backend monolith utilizing PostgreSQL for account history, and Redis for leaderboard management.

Challenges we ran into

The JS Bridge Bottleneck: Passing 30 high-resolution camera frames per second through the React Native JavaScript bridge caused severe frame drops, UI lag, and device overheating. We solved this by enforcing the "Camera Frame Isolation" rule, ensuring raw camera frames never leave the native layer.
Frame Drop Scheduling: Under thermal throttling, processing queues would back up. We implemented a "latest frame wins" scheduling algorithm—when the pose engine is busy, old frames are dropped, prioritizing the newest frame to keep feedback latency near 0ms.
Dynamic Environment Calibration: Home workout spaces have varying lighting, camera angles, and distance. We had to design a robust calibration filter (minimum 65% pose confidence, requiring 82% body coverage within the frame) before letting an exercise start.
Biomechanical Jitter: Tiny tremors or natural movement pauses near threshold values (e.g., squat depth limits) caused the state machine to jitter back and forth. We solved this by implementing state machine hysteresis and minimum duration thresholds to smooth transitions.

Accomplishments that we're proud of

Stable Edge-AI Performance: Achieving 15+ FPS real-time tracking on budget-to-midrange Android devices common in Indonesia.
100% Volatile Privacy: Creating a camera-based AI coaching experience that operates completely locally. Zero video frames or personal camera feeds are saved or uploaded to the internet.
Clinical Explainability: Building a fully auditable rule engine. Coaches and physiotherapists can read and adjust our exercise state machines without needing to retrain complex neural networks.
Superb Unit Economics: Reducing backend processing costs by 95% compared to cloud-based computer vision apps, proving the startup is highly scalable.

What we learned

Edge AI is Economically Vital: Moving compute loads to the client device isn't just about privacy; it's a financial game-changer that makes AI apps viable for bootstrapped teams.
Health Data Complexity: Health data from Health Connect (Android), HealthKit (iOS), and Huawei Health Kit have different structures. Building a normalized provider adapter was crucial.
Hysteresis in Physical Logic: Computers process discrete numbers, but human movements are continuous. State machines tracking human biomechanics must have error buffers (hysteresis) to prevent UI noise.

What's next for Pocket Trainer

iOS and Apple HealthKit integration: Bringing Swift Turbo Modules online and reading advanced readiness metrics from HealthKit and Apple Watches.
Insurtech & B2B Partnerships: Integrating our open APIs with health financing and insurance apps in Indonesia (like Rey.id) to reward users with premium discounts for injury-free workouts.
Digital Pharmacy Integrations: Syncing workout strain logs with e-pharmacies (like Lifepack.id) to recommend wellness supplements, joint braces, or recovery aids.
Expanding Exercise Library: Scaling our JSON-based exercise definitions from the core 4 hackathon exercises to over 100 gym, yoga, and physical therapy movements.