SOULSYNC

"Syncing Mind, Body, and Soul for a Healthier You."

Comment

🌟 Inspiration

Fitness trackers are broken. Most health apps behave like gloriously formatted spreadsheets—you manually log numbers, look at graphs, and get generic notifications. We realized that true physical and mental health are inseparably linked: your nutrition affects your workout stamina, your exercise improves your mental clarity, and your mental state dictates your diet.

We wanted to build SOULSYNC: a unified ecosystem that actually syncs your lifestyle. Instead of just manual logging, we wanted to build a real-time system that uses computer vision to track physical activity (your body), integrates with certified databases to manage diet (your nutrition), and uses conversational AI to support your mental well-being (your mind).

🛠️ How We Built It

SOULSYNC uses a modern decoupled architecture:

  • Frontend (The Interface): Built with React and Vite styled using TailwindCSS. We designed a dashboard with interactive goals, hydration trackers, and live session panels.
  • Backend (The Core): A lightweight Flask (Python) REST API server configured with JWT authentication to manage sessions and log user entries securely in a MongoDB database.
  • Computer Vision (The Tracker): A live MediaPipe Pose and OpenCV pipeline running locally on the user's device. We used trigonometry on joint landmarks (elbows, hips, shoulders, and head orientation) to detect rep counts and evaluate whether the user's back is straight or hips are misaligned in real-time.
  • Nutrition Intelligence: Integrated the USDA FoodData Central API for search, autocomplete suggestions, and full nutritional breakdowns (calories, protein, carbs, fats).

🚧 Challenges We Faced

1. The Child Process Freeze (OS Buffer Limits)

When launching the OpenCV camera tracker from the Flask backend, we originally redirected standard outputs using standard pipes. However, as the camera processed frames, it wrote debugging info faster than Flask consumed it. The OS pipe buffers filled up, and the camera process froze mid-rep. We resolved this by routing outputs to a dynamic logging directory (logs/push_counter_<session_id>.log) and replacing standard pipes with a file-based JSON IPC.

2. Real-Time Stats Sync

Because the OpenCV module runs in its own system thread/process to avoid blocking the main server, we couldn't easily pass variable values back to Flask. We created a lightweight file-based Inter-Process Communication (IPC) system where the tracking module periodically flushes statistics (reps, form accuracy, calories) to a temporary JSON file. Flask reads this file asynchronously to feed live stats back to the React UI.

3. API Parameter Serialization

The USDA API has strict query parameter serialization guidelines. Passing multiple data types as standard arrays resulted in duplicate keys in GET requests, throwing 400 Bad Request errors. We resolved this by rewriting our query-builder to feed data types as a single comma-separated query string ("Survey (FNDDS),Branded").

🏆 Accomplishments We're Proud Of

  • Zero-latency form feedback: The computer vision model successfully detects when a user is "not low enough" or has a "sagging back" during push-ups, rather than just blindly counting reps.
  • Crash recovery: If the backend server restarts mid-workout, the stats API auto-recovers and re-establishes the connection using the active session file.
  • Holistic design: Creating a workspace where workout stats, nutritional diaries, and water intake are visualized side-by-side in a responsive glassmorphism UI.

📖 What We Learned

  • We got hands-on experience tuning joint angle thresholds in computer vision models to make tracking strict enough to prevent injury, but lenient enough to allow for natural movement differences.
  • We learned how to handle asynchronous subprocesses and process redirection safely in Python without blocking main thread loops.

🚀 What's Next for SOULSYNC

  • WebRTC Migration: Move the OpenCV python tracker from a local window to a browser-based WebAssembly/WebRTC model so users don't have to launch separate desktop windows.
  • Expanded Workout Registry: Add pose tracking algorithms for squats, lunges, and plank durations.
  • Wearable Syncing: Integrate Apple HealthKit and Google Fit APIs to sync passive daily stats like step counts and heart rates.

Built With

Share this project:

Updates