Body Graph Scan

Inspiration

Everyone who cares about their health should be able to measure their body fat % without investing in expensive equipment. This project is inspired by the challenge of making clinical-grade metrics, like DEXA body fat analysis, accessible to anyone with a smartphone.

What it does

Body Graph Scan estimates body fat % and physical measurements from user photos of their body (a front view and a side profile), weight and height.

• User fills in gender, date of birth, height, and weight on profile creation
• User captures a front-facing and side-profile photo using the in-browser camera. A translucent silhouette & alignment grid overlay ensure consistent image capture
• Photos upload directly to Vercel Blob storage from the user's camera via browser
• AI graph pipeline breaks each image into isolated body regions (belly, waist, chest, hips, neck, jawline, triceps, forearms) and analyzes each one independently using a Vision Language Model (Qwen)
• Fan-in aggregation combines per-region estimates with configurable weights to produce a final body fat % and circumference measurements
• Results are stored so users can track their body composition trends over time with charts.

How we built it

• Non-linear Graph Architecture -- This is implemented using LangGraph, where each graph node represents a specific body region for parallel analysis.
• Swappable LLMs -- switch between Qwen, Gemini or other VL models per node
• Configurable weights -- control how much each body region contributes to the final estimate
• Training pipeline -- refine feature extraction and weights from labeled datasets (supervised learning)

Challenges we ran into

Asking a single VLM to analyze an entire body photo at once leads to hallucination and unreliable estimates. A single image is too broad an input and leads to vague output estimates.

I solved this by fanning out the image into focused nodes for each region, each node analyzed in isolation with managed context, and then fanning in the results with weights. This way we get significantly more accurate and reproducible data.

For inspiration I used the US Navy method of body fat estimation using the measurements of neck, waist, and hip circumferences combined with height. But we can finetune the weights of each specific body region based on labeled dataset (images with known body fat%)

Accomplishments that we're proud of

I'm particularly proud of the training loop for weight optimization. Instead of just hoping the AI is correct, we built an admin dashboard to upload labeled datasets of images with known body fat. We then use a coordinate-descent optimizer to adjust each body region node weights, to minimize the Mean Squared Error (MSE) against ground-truth body fat data.

What we learned

I learned that Qwen and other VLMs are more effective at relative visual estimate, eg. comparing a body part's size to the user's known height is easier than absolute guessing. Once we anchor the model's reasoning to a known variable (height), it's easier to measure other regions circumference.

I also learned that instead of overloading the model's context with a huge system prompt with guidelines of how to estimate the body fat %, it's more effective to slice the image into specific body parts and then individually analyze them for eg. neck circumference, waist ciscumference, etc. Then we can 'fan-in' or combine the individual body part analysis output into the final body fat %. Between all this, we manage the memory better by persisting each individual analysis in DB and reading again when needed for final analysis. This works better instead of keeping it all in the model's context because that can lead to hallucinations.

What's next for The Body Scan

Apply for PoC funding to conduct a larger clinical validation study for training and extend the product to native iOS and Android apps.

Currently, I only built a webapp for the MVP due to hackathon's short timeline. For user adoption, having Android and iOS native apps would be better. Using a crossplatform React Native or Flutter framework would be next technical steps.

We would need more training dataset of images with known body fat% from other methods like DEXA scans. With training we would be able to refine the weights of individual body parts and maybe even discover correlation of body fat% with other body parts measurements. That would lead to additional feature discovery which could be added as more nodes to the graph pipeline for analysis.

We would also be able to train for specific regional/ethnic body proportions in future versions instead of just male vs female in current version.

Built With

• langgraph
• neonauth
• next.js
• node.js
• postgresql
• qwen
• typescript
• vercel