BodyLens

Accurate Body Measurement Using Two Photos with Qwen AI

Comment

Inspiration

Finding accurate clothing sizes online is still frustrating. Most people only know rough measurements, size charts vary widely between brands, and shopping often turns into trial-and-error with returns.

For me, this problem is personal. I’ve always struggled to shop for clothes online because I don’t actually know my exact size. Every purchase feels like a guess—sometimes it fits, often it doesn’t. It’s inconvenient, time-consuming, and honestly discouraging.

At one point, I went to the gym last month and had a full body analysis—body fat, muscle mass, and other metrics. The process required specialized handheld equipment and took time. While the results were useful, it left me with a bigger question: why is understanding your own body still so inaccessible? How can ordinary people estimate and track these measurements regularly, easily, and without relying on expensive devices?

What I really wanted was a simple way to monitor my body measurements anytime—without needing to visit a gym or medical facility. Just something practical that works with what most people already have: a mobile phone.

When I saw the Elfie Challenge #2 in this hackathon, it immediately clicked. This wasn’t just a technical challenge, it was an opportunity to explore a problem I genuinely care about.

That’s why I built BodyLens, to make body measurement collection fast, private, and practical using just two photos and a known height.

What it does

BodyLens estimates core body measurements (waist, chest, hip, neck, shoulder), computes body-fat percentage (US Navy method when inputs are available), and returns a confidence-aware result with quality signals and warnings.

Users upload front and side photos plus height, then receive:

  • structured measurements in centimeters
  • uncertainty per measurement
  • image quality and input-validity review
  • an explainable result source (geometry-only, AI-assisted, or rejected for low quality)
  • history tracking for each submitted estimate

The system is designed to reject unsafe inputs rather than produce overconfident numbers.

How we built it

BodyLens uses a hybrid architecture:

  • Geometry Engine (Python Worker / FastAPI): my deterministic measurement core.
    • MediaPipe Pose: detects body landmarks from front and side photos for height calibration, body alignment checks, and region localization (neck/chest/waist/hip).
    • MediaPipe Segmentation: provides baseline body masks for contour extraction and direct geometric width/depth measurements.
    • SAM2 branch: generates additional mask-driven candidates (coarse, torso, fine) to improve contour coverage when baseline masks are weak.
    • Measurement candidate stack: computes multiple candidates (direct geometry, SAM2-based geometry, ratio-based, and body-profile priors), then compares them instead of trusting a single formula path.
    • Reliability metrics: segmentation quality, landmark coverage, candidate agreement gap, and per-field confidence are used to score each candidate and estimate uncertainty.
  • Node API (Express): orchestration, validation, history persistence, confidence scoring, and final response shaping.
  • Qwen-VL integration (critical layer): this is the part that can actually "see" the uploaded photos and perform semantic judgments that pure geometry cannot.
    • Visual understanding: inspects the real front/side images directly, not just numeric landmarks.
    • Input validation: judges whether each view is truly usable (full body visibility, true side profile, posture quality, lighting, occlusion).
    • Pair consistency checks: verifies front/side are the same person and flags invalid pairs.
    • Clothing intelligence: classifies garment tightness by region (neck/chest/waist/hip/shoulder), which is used to correct measurement bias from loose clothing.
    • Independent measurement + quality reasoning: provides a second estimate with explicit quality issues and confidence, then feeds fusion decisions.
  • Fusion logic: combines Geometry Engine evidence and AI signals, applies clothing correction and uncertainty modeling, and can reject low-quality image pairs.

I also implemented layered logging and fallback execution (HTTP worker + direct Python mode) to keep processing robust in hackathon conditions.

Challenges we ran into

I ran into four major challenges:

  • No single tool could solve the full problem reliably. Geometry, SAM2, and vision-language models each have different failure modes, so using only one of them produced blind spots. This is why I combined multiple tools and multiple validation layers instead of depending on a single estimator.
  • Lack of ground-truth data. Within hackathon constraints, I did not have enough time and resources to collect a sufficiently large labeled dataset, which is critical for evaluating correctness and real accuracy.
  • Time constraints on exploration. I wanted to test more vision models, reasoning models, and additional geometry approaches, and I also wanted to prototype a video-upload pipeline (not only 2 photos), but I could not complete all of that in the available time.
  • Variability in pose, lighting, camera angle, and loose clothing caused unstable contours.

Accomplishments that we're proud of

  • Built a full end-to-end measurement pipeline with deterministic vision + multimodal AI fusion.
  • Integrated SAM2-derived measurement candidates into the evidence stack with confidence gating and fallback behavior.
  • Made Qwen-VL a first-class safety and quality layer for photo validation, clothing judgment, and cross-image consistency checks.
  • Added transparent quality diagnostics so users understand when results are reliable.
  • Implemented correction logic for clothing looseness and generated uncertainty, not just point estimates.
  • Designed the backend for production-minded behavior: async processing, error-safe responses, and result history.

Most importantly, I shipped a version that prefers trustworthy output over flashy but unreliable numbers.

What we learned

  • I conducted industry and competitor research to validate market need and benchmark how leading players perform, including 3DLOOK, Nettelo, Meepl, Fit Analytics, Size Stream, Styku, and Bodygram.
  • Measurement estimation is as much an input-quality problem as a modeling problem.
  • Deterministic geometry gives stability; multimodal AI adds context and validation.
  • Confidence without explainability is risky in body-related applications.
  • Guardrails (quality checks, consistency checks, rejection criteria) materially improve user trust.

What's next for BodyLens

  • Enlarge the ground-truth dataset with more subjects and more variation in pose, clothing, camera angle, and lighting.
  • Explore additional geometry methods and additional vision-language models to compare robustness, correctness, and consistency.
  • Investigate a self-trained model specialized for this exact body-measurement problem.
  • Provide a mobile app SDK with input-quality control to guide users before capture/upload.
  • Continue researching market needs and launch opportunities to identify the best path to product-market fit.
  • Improve measurement accuracy with larger calibration datasets and benchmarked error metrics per body region.
  • Add guided capture UX (live framing, posture hints, clothing recommendations) before photo submission.
  • Expand outputs for practical use cases: fit profiles, brand size mapping, and change-over-time tracking.
  • Continue privacy-first architecture with stronger data retention controls and optional on-device preprocessing.
  • Introduce evaluation dashboards to compare geometry-only vs fusion performance over real-world samples.

Built With

Share this project:

Updates