Newfoot

A camera reads your stance and designs a corrective insole: Better posture, fewer aches, from the ground up. Benefit: - Fix your stance, spare your knees, back, and neck. - Stand taller, hurt less.

Comment

💡 Inspiration

Everyday aches — sore knees, a stiff lower back, a tight neck — often trace back to something invisible: how we stand. A tilted pelvis, rounded shoulders, or an over‑pronating foot quietly loads the joints wrong, and a well‑designed insole can help correct it. But getting there today means a clinic visit, a scanner kiosk, an app, or a two‑week mail‑in kit.

We wanted to collapse all of that into a single link. Newfoot turns any phone camera into a posture lab — read your stance, understand what's off, and design a corrective 3D‑printed insole — with a hard rule: your video never leaves your device.

🦶 What it does

  • Scan — an on‑device pose model draws a real‑time 33‑point skeleton over your live camera, and overlays an ideal reference posture beside your real one.
  • See — it computes posture findings (shoulder tilt, pelvic tilt, head lean) with severity, in plain language — to help reduce knee, back, and neck strain.
  • Solve — a rule engine turns your findings + foot size into a corrective insole: arch support, heel posting, stiffness, and material — shown as an interactive 3D model and exported as a printable STL.

Wellness/visualization POC — not a medical device; a disclaimer is on every screen.

🛠️ How we built it

  • Frontend: Angular 22 + MediaPipe Pose Landmarker running 100% in the browser (WASM in a Web Worker), a multi‑rate perception pipeline (capture → inference → smoothing → metrics → render) that keeps the UI at 100+ Hz while inference runs off‑thread, and Three.js for the 3D insole.
  • Backend: NestJS + MySQL, dockerized end‑to‑end; one command brings the whole stack up.
  • Sponsor infra: deployed on Akash (decentralized compute, via console-axi managed wallet — no private keys); the landing page is hosted free on Zero (the agent CLI); Pomerium as the identity‑aware front door.
  • The twist — an autonomous build: the app was assembled by an agentic loop we wrote — one feature per session, each on its own branch → PR → auto‑merge, fully auditable (20+ PRs). We also designed a self‑improving analysis loop that tunes the posture thresholds against self‑supervised signals, with deterministic fixtures as a guardrail.

🧗 Challenges we ran into

  • Cross‑origin isolation. MediaPipe's threaded WASM needs SharedArrayBuffer, which silently works in Chromium but fails elsewhere without COOP/COEP headers. Our tests were green while the real browser showed "pose engine failed to load" — a classic Chromium‑only blind spot. We added the headers and a smoke test that asserts crossOriginIsolated.
  • Trusting synthetic input. A fake camera can't produce a real skeleton, so we verified pose math with fixed landmark fixtures and drove the browser with a real full‑body clip.
  • One repo, one writer. The autonomous loop and manual edits collided on the git tree until we enforced a strict single‑writer rule and a branch → PR → merge contract.
  • Shipping on a clock. With a hard deadline we cut scope hard to a clean vertical slice (scan → findings → insole → 3D → STL) and let the deeper features live as documented roadmap.

📚 What we learned

  • On‑device inference is a feature, not a constraint — "your video never leaves your device" is both the privacy story and the fastest path.
  • Loops beat snapshots for real‑time analysis: temporal fusion + persistence gating turn jittery angles into trustworthy findings.
  • Agentic engineering scales when you give it durable state (a feature list, a progress journal, ADRs) and a hard verification contract.

Built With

Share this project:

Updates