Vital_Lens

Inspiration

Severe acute malnutrition (SAM) affects millions of children under 5 worldwide, particularly in remote and low-resource settings. Community health workers (CHWs) and caregivers still rely on disposable paper MUAC (Mid-Upper Arm Circumference) tapes that tear easily, create recurring procurement costs, and are frequently out of stock at rural clinics.

I wanted to ask: Can a standard smartphone camera become an instant, offline, zero-marginal-cost malnutrition screening tool?

I built VitalLens to bridge this critical triage gap—enabling frontline health workers and parents to screen children (6–59 months) in seconds using a single photo and a reusable 5 cm printed calibration square, without sending any sensitive health photos to cloud servers.

What it does

VitalLens is a mobile-first Progressive Web App (PWA) that estimates child MUAC directly in the browser:

• Instant On-Device Calibration: The user holds a reusable 5.0 cm black calibration square against the child's upper arm.
• Computer Vision Pipeline: The app detects the calibration square using rotation-invariant central moments, calculates the precise pixel-to-centimeter scale, and measures apparent arm width via center-outward radial gradient edge detection.
• MUAC Estimation ($C \approx \pi \times \text{width}$): Approximates arm circumference and classifies nutritional status against WHO 2013 Child Malnutrition Standards (6–59 months):
  • 🔴 Red (SAM): MUAC < 11.5 cm — Immediate referral to a healthcare facility.
  • 🟡 Yellow (MAM): 11.5 cm ≤ MUAC < 12.5 cm — Supplementary feeding & 3-day triage.
  • 🟢 Green (Normal): MUAC ≥ 12.5 cm — Normal nutritional range.
• Frontline Accessibility: Includes bilingual voice guidance (English & Hindi), high-contrast accessibility mode, and offline installation.
• CHW Workflow & Data Security: Complete local history stored in IndexedDB, CSV data export with formula injection protection, and printable one-click clinical referral sheets with clinic signature lines.

Disclaimer: VitalLens is an optical screening and triage aid, not a diagnostic medical device. It assists in prioritizing children for formal physical confirmation.

How I built it

As a solo developer, I engineered the entire software stack to run 100% on-device without external server dependencies or heavy OpenCV WASM runtimes:

1. Lightweight Computer Vision (Custom TypeScript/Canvas):
   • Otsu Dark Thresholding & CCL: Rapid segmentation of the high-contrast calibration square.
   • 2nd-Order Central Image Moments ($\mu_{20}, \mu_{02}, \mu_{11}$): Computes orientation angle $\theta = \frac{1}{2}\text{atan2}(2\mu_{11}, \mu_{20}-\mu_{02})$ and rotation-invariant side length $s = \sqrt{A} \approx \sqrt{6(\mu_{20}+\mu_{02})}$, ensuring accurate readings even when the card is tilted against an active child's arm.
   • Radial Gradient Edge Consensus: Multiple transverse scanlines along the arm axis find skin-to-background boundaries, using statistical median consensus to reject lighting artifacts and clothing wrinkles.
2. Frontend & Mobile UI: React 19, TypeScript, Tailwind CSS, and Vite. Designed with touch-first ergonomics, safe-area viewport insets, and high-contrast accessibility themes.
3. Storage & Offline Capabilities: Client-side IndexedDB with idb, RFC 4122 v4 UUID fallbacks for local network testing, and Service Worker caching via vite-plugin-pwa.
4. Multilingual Speech & Printing: Web Speech API for voice assistance in Hindi and English, and an iframe-fallback printing engine that bypasses mobile popup blockers.

Challenges I ran into

• Rotation and Perspective Distortion: Initial bounding box approaches failed when the card was held at a 20°–30° angle, inflating scale by ~28% and artificially shrinking arm measurements into false SAM classifications. I solved this by deriving principal axes and second-order central moments directly from raw image pixels.
• Skin Tone and Background Contrast: Outdoor field conditions vary from bright sunlight to dim indoor huts. I developed a center-outward radial search algorithm rather than a global gradient extrema search to ensure background shadows and textiles don't register as arm contours.
• Mobile Hardware Constraints: Making sure the camera feed, image scaling, and real-time canvas analysis execute smoothly in low-end smartphone WebViews without memory leaks or battery drain.

Accomplishments that I'm proud of

• Built a fully functioning, zero-latency computer vision pipeline entirely in vanilla TypeScript that runs in ~100ms on mobile browsers without requiring heavy machine learning models or internet connectivity.
• Maintained 100% privacy: No health data or child photos ever leave the device.
• Created an end-to-end practical tool for community health workers, complete with bilingual audio guidance and printable referral slips.
• Built, debugged, and verified the entire project as a solo developer with unit tests and production PWA support.

What I learned

• Mathematical formulation of image moments and eigenvalues for rotation-invariant shape extraction.
• The real-world operational constraints faced by community health workers in triage scenarios.
• Nuances of cross-browser Web APIs on mobile, including Web Speech utterance lifecycles, non-secure context Web Crypto fallbacks, and mobile print sandboxing.

What's next for VitalLens

• Multi-Angle / Video Sweep: Incorporating 3D elliptical reconstruction from a 2-second video pass around the arm to account for non-circular arm cross-sections.
• ArUco / AprilTag Reference Cards: Adding coded fiducial markers to the printable card for sub-millimeter pose estimation.
• Pilot Partnerships: Collaborating with local humanitarian NGOs and public health clinics to conduct clinical validation against standard Shorr tape measurements.

Built With

• canvas-api
• computer-vision
• html5
• image-processing
• indexeddb
• javascript
• mobile
• moment-invariants
• otsu-thresholding
• pwa
• react
• tailwind-css
• typescript
• vite
• web-speech-api