Capture setup: athlete, movement, camera, pain check and consent before any recording.
Lab sheet: a still, frame, timestamp and confidence for every measurement. Coarse timing is declared below 120 fps.
Anatomy Lab: modeled muscle-demand map per pitch phase. Screening and education only, kept separate from measurements.
Real pipeline keyframe: stride-foot contact (frame 448) on a consented take.
Real pipeline keyframe: max external rotation, 2D proxy (frame 450).
Kinetech AI: evidence-led movement screening and coaching support.
Film Room: the take with the kinematic-chain skeleton, 3D view and a Findings card that says when a check could not run.
Real pipeline keyframe: ball release (frame 457).

Kinetech AI: evidence-linked movement screening and coaching support, starting with baseball pitching.

Live app: https://kinetech-ai.vercel.app · Seeded demo review: https://kinetech-ai.vercel.app/sessions/bf43bf73-6b41-4733-ba97-2a95d9fc91ec/review · GitHub: https://github.com/ccocokios-ctrl/Kinetech-AI

The problem

Throwing-arm overuse is one of the most common problems in youth and amateur baseball. The tools that actually measure pitching mechanics are motion-capture labs that cost hundreds of thousands of dollars, or phone apps that print a single black-box "score" with no evidence behind it. Most coaches, parents and athletes have neither. They cannot see which movement patterns research associates with extra strain on the elbow and shoulder, how confident the measurement is, or when it is time to ask a clinician. Kinetech is a screening and coaching aid. It is not a diagnosis and not an injury predictor, and the product refuses to pretend otherwise.

Our solution

Kinetech turns a laptop camera, an iPhone (live over Continuity Camera, or as the recorder through a one-time QR link), or an imported high-speed clip into an evidence-linked movement review. Consent is recorded first. The clip is quality-checked and normalized to a constant frame rate, then a Python pose worker runs real pose estimation (MediaPipe Pose Landmarker), detects the pitching phases (leg lift, stride-foot contact, max external rotation, ball release, follow-through), computes 2D biomechanics metrics with per-frame confidence, and applies versioned, population-scoped rules against an evidence ledger of 60 cited studies. A full analysis of a take runs in about 20 seconds on production.

The Film Room shows the result: the video with the skeleton overlay, a levelled 3D replay with camera presets, joint-angle charts that seek the video on click, and a Lab sheet with a still, frame number, timestamp and confidence for every measurement. Every metric card shows your result against a published reference range and says plainly when no validated range exists for a single-camera estimate. Findings separate four layers: what was observed, what research associates it with, one coaching cue, and when to consider review by a qualified clinician. A PDF report with the stills, rule checks, lab sheet and citations exports in one click.

Coach K is the in-product teammate: a voice coach you talk to with the mic on by default. He explains results, answers pitching, anatomy and research questions with cited evidence IDs, opens the command center at the exact source frame a number came from, and operates the platform by voice ("export and show me the PDF", "open the anatomy lab", "compare my last two attempts"). He can only speak from the validated, structured analysis: a validator rejects any spoken or written sentence containing a number that is not in the pipeline output.

Challenge entered

HarborHack 2026, Agent In The Loop. Non-humans were real teammates in two places: inside the product (Coach K), and building it (a Claude Code "chair" session dispatching parallel builder agents, with Codex doing research and adversarial review). The humans owned product direction, the safety and medical-language rules, the consented recordings and the demo. Details are in the collaboration story below.

What makes it different

Honesty is built into the system, not the copy. A capture-quality tier stops the app from claiming timing it cannot measure at 30 fps. Every rule carries a population scope, so a threshold measured on professional pitchers is never applied as a flag to a 12-year-old. Reference bands measured with marker-based 3D systems are shown as context, never as a flag, on a single-camera run.
The evidence is visible. Source frames, confidence and citations on every finding instead of one "injury risk" number.
It runs on hardware people already own, and it is multi-tenant from day one: every row carries an organization, row-level security on every table, private storage with signed URLs, consent gates in the database, and an append-only audit log.
The demo is the product. The seeded offline mode serves frozen output of the real pipeline on a consented clip (labelled SYNTHETIC athlete), with checksums verified before every deploy. We fake the scaffolding, never the claim.

Repository access

The GitHub repository (https://github.com/ccocokios-ctrl/Kinetech-AI) is currently private to the team owner's account. If it is not public by judging time, judges can request access from the owner (ccocokios-ctrl). The live app, the seeded demo review and the PDF report above are open without a login.

Non-human collaboration story

Who was on the team

Humans: Vonzelle Brown and the repo owner (ccocokios-ctrl). They set the product direction, made every safety and medical-language call, recorded the consented takes, and owned the demo and final review.
Claude Code, the build "chair": one long-running session that planned, specified, integrated, ran migrations and gates, and deployed. It split the work into 21 PRDs with collision-free file manifests and dispatched background builder agents in parallel (metrics engine, rules engine, review workspace, report and voice, capture, Coach K, demo mode, the polish program), then checked each one's output on disk before committing. The git history shows 166 commits in about 42 hours.
Codex (a second model family): research and adversarial review: the competitive teardown, the safety framing, the iPhone-capture architecture review (server-bound one-time tokens, HDR handling, provenance), and the stylistic "anti-slop" gate before commits.
Coach K, the in-product teammate: an ElevenLabs voice over a grounded response contract. He can only speak from the validated, structured analysis, and he drives the UI through an allowlist of typed actions (seek to a frame, open a view, export the PDF).

How responsibilities evolved

Plan first. The humans described the product. The agents turned that into research with citations, the PRDs and an evidence ledger before any code. The humans then cut scope to one demo path.
Parallel build. The chair stopped writing most code itself. It wrote briefs (files in scope plus a machine check), ran up to four builders at once in separate worktrees, and became the integrator and verifier.
Mid-build briefs without stopping. The humans kept sending new requirements from real use: any-angle capture ("facing direction must never gate recording"), iPhone as camera or recorder, reference ranges on every card, a Coach K that never says "I don't have information", and mic-on-by-default conversation. The chair folded each into specs while builders were running and deferred anything that would collide with their files.
Audit, then ship. On the last morning the owner asked for a whole-product audit. The agents audited every page, wrote a ship spec with 13 units and one machine-checkable oracle each, and landed them in a few hours.

Where the non-humans changed the design

Grounding as a hard rule. The agents proposed, and the humans adopted, a validator that rejects any sentence containing a number not found in the pipeline output. The narration, the cards, the PDF and Coach K all render one structured report.
Population fairness. An agent flagged that our thresholds came from studies of professional or college pitchers. Applying them to a 12-year-old would be extrapolation. Every rule now carries a population scope, and when the population does not match, the app shows descriptive output instead of a flag.
System-scoped references. Codex noticed the stride-length band came from marker-based 3D lab data. On a single camera the band is now shown as context with the mismatch stated, and the demo lost its only "finding" as a result. The owner chose honesty over a more dramatic demo.
Capture honesty. Research showed browser webcams top out around 30–60 fps while arm rotation lasts about 30–50 ms. So the app shows a "Coaching vs Biomechanics" quality tier and suppresses timing metrics it cannot measure.

What surprised us

The real LLM's report text (NVIDIA Nemotron) failed our own validator: it invented comparisons and rounded numbers. The deterministic template won. The safety net we designed caught the model we had planned to rely on.
Coach K was hearing himself. On production the realtime speech provider was refused (no credits), so every conversation ran on browser speech recognition, which hears Coach K through the laptop speakers. His own echo barged in and became fake conversation turns, and a default system voice read fallback lines. The fix treats an aligned prefix of his own words as echo and lets only the athlete's tail of the transcript become the turn. The tests use the production transcripts verbatim.
A thread count moved the science. Resizing the pose worker changed the x264 encoder's thread count, which changed pixels in the normalized copy, which moved ball release by hundreds of frames and changed 9 of 14 metrics. The encoder threads are now pinned and the pipeline version is bumped whenever a pixel can change.

What did not work, and what we changed

Shared working tree. Two builder agents ran git stash in the same checkout at the same time. Nothing was lost, but every later brief forbids stash, reset and checkout, and the polish program gave every unit its own worktree.
Secret handling. An agent loaded .env.local with source; a token containing a space broke the parse and part of it was echoed into a local transcript. We switched to reading one key at a time and scheduled key rotation after the event.
Hand gestures. A MediaPipe gesture prototype controlled the HUD touchlessly. It worked in testing and was removed on the last morning: it competed with the mic for the camera and added false positives the demo did not need.
Naming. The coach was renamed mid-build. Keeping user-facing copy separate from code identifiers made the rename a small change.

What we would change

Isolated worktrees per agent from the start, production smoke tests in CI rather than by hand, and real athlete clips on day one. Until the first consented take on Saturday morning, phase detection and auto-capture were tuned on synthetic landmark replays, and the first production run found three blockers that no unit test could have.