FieldMD

• 

  What it is

• 

  How it works

• 

  Feedback

• 

  biomechanical analysis

• 

  preferences

Inspiration

An estimated 5.6 million sports injuries affect children and young adults in the United States every year, according to research cited by the CDC. Sports and recreation injuries also account for approximately 2.7 million annual emergency-department visits among people ages 5–24.

Yet access to sports-health support remains deeply unequal. The National Athletic Trainers’ Association reports that only 37% of U.S. high schools have access to a full-time athletic trainer.

That creates a massive gap. Movement patterns involving balance, knee control, trunk stability, and left-to-right asymmetry may never be measured—not because movement science does not exist, but because the equipment and expertise are inaccessible to many youth athletes, school teams, recreational athletes, and low-resource communities.

We built FieldMD around one ambitious question:

What if meaningful movement screening were as accessible as opening your phone camera?

FieldMD does not claim that a camera can diagnose an injury or replace a clinician. Its purpose is to make the first layer of objective movement awareness dramatically more accessible.

What it does

FieldMD turns a phone or laptop camera into a real-time movement-screening tool.

The user chooses an assessment, follows guided positioning instructions, and performs the movement in front of the camera. FieldMD tracks 33 anatomical body landmarks, calculates biomechanical measurements locally, validates the movement through an assessment-specific state machine, and converts the measurements into understandable findings.

The current prototype supports two complete assessments:

Single-Leg Squat

FieldMD evaluates:

• knee tracking relative to the ankle
• squat depth
• pelvic stability
• trunk lean
• repetition validity
• left-versus-right differences

The assessment requires five valid repetitions. Incomplete movements, low-confidence frames, and camera noise are rejected rather than counted automatically.

Single-Leg Balance

FieldMD evaluates:

• torso-normalized postural sway
• trunk movement
• hold completion
• foot-down and stability-loss events
• pose loss
• left-versus-right differences

A valid hold must be completed on both anatomical sides before the session can finish.

After an assessment, the user receives:

• a clear primary observation
• left-versus-right comparison
• the measurement behind each major finding
• a plain-language explanation
• general movement-practice guidance
• locally saved history
• progress comparisons across repeated sessions

FieldMD is a movement-screening and educational prototype. It is not a medical device, does not diagnose injuries, and does not predict whether an injury will occur.

What I'm proud of

movement screening anywhere a camera exists.

Detailed motion analysis is often concentrated in sports laboratories, clinics, universities, professional organizations, and well-resourced training facilities. Meanwhile, millions of athletes train in schools, homes, community gyms, and recreational programs without consistent access to objective movement feedback.

FieldMD explores a future in which a youth athlete, school coach, rehabilitation patient, or recreational athlete can perform a repeatable movement screen using equipment they already own.

The larger vision is not to replace clinicians. It is to close the enormous gap between receiving no objective movement feedback and receiving professional evaluation.

The camera is already in millions of pockets. The Moonshot is building reliable, explainable, and accessible movement-screening infrastructure around it.

How we built it

FieldMD uses a layered and explainable processing pipeline:

Camera → Pose Tracking → Anatomical Mapping → Geometry → Assessment State Machine → Deterministic Findings → Movement Guidance

Real-time pose tracking

MediaPipe Tasks Vision detects 33 body landmarks directly inside the browser.

The camera footage and raw landmark stream remain on the user’s device. The analysis does not require uploading video to a server.

Anatomical mapping

Front-facing video is visually mirrored, but biomechanical assessment must still refer to the user’s true anatomical left and right.

FieldMD separates display coordinates from anatomical meaning so that instructions, measurements, and reports consistently refer to the correct side.

Deterministic biomechanics

Instead of asking a language model to interpret raw video, FieldMD calculates measurable features using geometry.

These include:

• knee flexion
• knee alignment relative to the ankle
• squat depth
• pelvic tilt
• trunk lean
• hip and shoulder sway
• hold duration
• stability-loss events
• left-versus-right symmetry

Measurements are normalized using body proportions where appropriate, reducing sensitivity to camera distance and body size.

Assessment-specific state machines

Detecting a person is not the same as validating a movement.

The squat assessment identifies standing, descent, bottom position, ascent, and repetition completion. A repetition must cross the required movement states before it is counted.

The balance assessment handles calibration, foot-lift detection, valid holding, foot-down events, pose loss, side completion, and final completion.

This allows FieldMD to reject incomplete or unreliable movement instead of producing a result from insufficient evidence.

Explainable findings

FieldMD uses fixed, transparent, code-defined thresholds.

Each major finding can be traced back to a measurement. Rather than presenting only an unexplained score, FieldMD tells the user what was measured, which side differed, and why the result may deserve attention.

These thresholds are part of the prototype and require future validation. They are not presented as clinical diagnoses.

Privacy-first product

• Video never leaves the device.
• Raw landmarks are not uploaded.
• No account is required.
• Profiles, sessions, history, and progress remain local.
• The application is installable as a Progressive Web App.
• Core functionality remains available after required assets have been cached.

Product stack

• TypeScript
• Vite
• MediaPipe Tasks Vision
• Canvas 2D
• HTML and CSS
• Vitest
• Progressive Web App technologies
• Vercel
• GitHub

Challenges we ran into

Validating movement instead of merely detecting it

Displaying pose landmarks was only the beginning. The difficult part was distinguishing a real repetition or balance hold from camera jitter, small body movements, incomplete attempts, and temporary pose loss.

We addressed this with explicit state machines, confidence gates, hysteresis, validity requirements, and session-level checks.

Correct anatomical left and right

The front-facing camera appears mirrored, creating a difference between what the user sees and what the landmark model labels anatomically.

An early version could associate a trial with the wrong side. We separated visual mirroring from anatomical interpretation, corrected the mapping, and added regression tests.

Preventing false balance completion

During early testing, a balance session could finish before both valid side trials existed.

We rebuilt the session lifecycle so completion requires valid measurements for both sides, matching session data, finite values, and complete result objects.

Converting measurements into understandable feedback

Values such as normalized displacement, tilt, or sway are useful to engineers but difficult for athletes to interpret.

We designed a reporting layer that preserves the real measurement while translating its meaning into concise, non-diagnostic language.

Creating a reliable mobile experience

Camera positioning, safe areas, overlays, touch targets, instructions, reports, and navigation all had to function on both phones and laptops.

Accomplishments that we are proud of

• Built a working real-time computer-vision product rather than a mocked demonstration
• Created two separate, functioning movement-assessment engines
• Implemented deterministic and explainable biomechanics instead of black-box scoring
• Built validity checks that reject incomplete or unreliable movements
• Kept video and raw movement data on-device
• Corrected anatomical left/right behavior for mirrored camera feeds
• Added locally stored assessment history and progress comparison
• Made FieldMD installable as a Progressive Web App
• Deployed the complete product publicly
• Finished with a strict TypeScript build and 33 passing regression tests

What we learned

The hardest part of computer vision was not drawing a skeleton.

It was deciding when the system had enough trustworthy evidence to make a conclusion.

We learned that credible movement analysis requires:

• reliable session state
• confidence handling
• anatomical consistency
• movement-specific protocols
• normalized measurements
• transparent findings
• safe failure behavior
• understandable communication

We also learned that technical complexity only creates value when a first-time user can understand what the system is measuring and why it matters.

What’s next for FieldMD

The current prototype proves the core mechanism using Single-Leg Squat and Single-Leg Balance assessments.

The next step is validation, not unsupported medical claims.

Future work includes:

• comparing measurements with laboratory motion-capture systems
• validation with athletic trainers and physical therapists
• personalized baselines
• drop-landing and lateral step-down assessments
• sport-specific screening protocols
• longitudinal progress across seasons
• optional dashboards for coaches and clinicians
• improved accessibility
• multilingual guidance
• multi-camera or depth-assisted analysis

Our long-term goal is to make objective movement screening more accessible without pretending that a phone replaces professional medical care.

Built With

• canvas-api
• computer-vision
• css
• html
• mediapipe-tasks-vision
• progressive-web-app
• typescript
• vercel
• vite
• vitest