Coachpoint

• 

• 

• 

• 

• 

• 

• 

• 

Inspiration

Home exercise care continues long after a patient leaves the clinic. Yet finding suitable exercise materials, assembling a program, explaining it, and understanding what happened between visits all take time.

Our starting point was Crosspoint's existing exercise library and home-exercise product concepts. We saw an opportunity to connect two recurring tasks: preparing a clear program before the patient leaves, and making sense of their practice afterward.

CoachPoint asks a practical question: What if an agent could handle more of the preparation and follow-up work, while the therapist kept every prescription decision?

What it does

CoachPoint connects a therapist, a patient, and a browser agent through one home-exercise workflow.

1. The therapist creates the plan. They select a client and describe their goals, available time, equipment, and proposed exercises in plain language. Through WebMCP, the agent uses the curated movement library and its safety notes to assemble a visible, editable draft. The therapist adjusts dosage and order, then confirms the prescription in the interface. There is no tool that lets the agent confirm it.

2. The patient practices and checks in. They open the confirmed program and choose when to start the camera. For the supported half-squat exercise, browser-local pose processing counts repetitions and provides immediate visual feedback, with optional audio cues. After the set, the patient reports perceived effort and pain. The agent can then explain the saved result and offer an optional focus for the next set. The patient accepts or dismisses it; the prescribed exercise and dosage remain unchanged.

3. The therapist reviews the follow-up. Back in the client workspace, the therapist can see recorded completion, repetitions, effort, pain, and camera-derived observations, then ask for a concise pre-visit summary grounded in those same records.

Why WebMCP fits this problem

The useful context is already inside the browser: the current client workspace, the editable draft, and movement results derived on the patient's device.

WebMCP gives the agent a structured way to work with that context through tools exposed by the current page. It can create something the therapist can immediately inspect, or retrieve a saved result without receiving camera footage.

A server-side integration would need an additional path to access our unsynchronized browser-local state. WebMCP lets us expose selected data and validated actions where that state already lives. The browser is both the shared workspace and the source of the movement observations.

How we built it

We built the application with Next.js, React, TypeScript, and Tailwind CSS, and deployed it on Vercel. MediaPipe pose estimation runs locally with a self-hosted model and WebAssembly runtime.

The human interface and WebMCP tools reuse the same domain operations for catalog search, case validation, dosage limits, time estimation, and draft creation. This keeps agent actions subject to the same checks as manual actions.

Tools register through document.modelContext.registerTool() and belong to the current route. An AbortController retires them when that route is left. Confirmed prescriptions are preserved as versioned snapshots, and patient results are validated before they become available for review. Representative tools include draft_program, review_completed_set, stage_next_set_focus, and get_adherence_summary.

The pre-existing library and design foundation are distinguished from the new prescription workflow, WebMCP integration, local motion processing, and feedback loop in our challenge-work documentation.

Challenges we ran into

Finding the right moment for the agent. We initially explored live monitoring and frequent spoken feedback. The interaction felt poorly paced. We separated immediate browser feedback from post-set reasoning: counting and short cues happen locally, while the agent interprets a saved result when the user asks.

Making assistance feel useful, not procedural. Early drafting involved too many separate tool calls and overly detailed prompts. We consolidated the common drafting path into one operation for an explicitly requested new draft. Existing drafts remain protected from silent replacement.

Keeping state and responsibility clear. Route changes, cancelled calls, interrupted camera sessions, and repeated actions could otherwise leave the interface and agent describing different situations. We added lifecycle cleanup, versioned persistence, conflict checks for coaching suggestions, and explicit human controls. We also kept detected repetitions separate from self-reported effort: eight repetitions and an effort rating of eight are different facts.

Accomplishments we're proud of

We built and rehearsed the complete loop: an agent-generated draft, a therapist's change from ten to eight squat repetitions, a completed camera-tracked set, an explicit effort-and-pain check-in, an accepted coaching focus, and a therapist-facing summary.

Automated tests cover domain rules, tool contracts, cancellation, persistence, and motion counting. Browser testing checks that tool results and the visible interface agree. The manual workflow remains usable without an agent.

What we learned

The most useful unit of agent work was a reviewable draft or a completed-set explanation, not a stream of individual clicks or camera frames.

We also learned that boundaries belong in the product itself. Therapist confirmation, patient acceptance, limited data exposure, and clear descriptions of camera estimates matter more than a disclaimer added at the end.

What's next for CoachPoint

The current build is a synthetic demonstration with three demo clients and fifteen catalog exercises. Camera-based counting currently supports the half squat; other exercises use camera-independent workflows. Data is stored in the current browser, so patient links are not yet a cross-device sharing system. The recorded demo uses OBS replay for repeatability. Camera observations are two-dimensional estimates, not clinical conclusions, and the product has not been clinically validated.

Next, we want to validate additional exercise-specific motion rules, test diverse real-camera conditions with clinicians, and build secure, consent-based cross-device access. We also want to measure preparation time and follow-up usefulness in practice before claiming productivity or clinical gains.

Less preparation work. More informed conversations. Clinical decisions stay human.

Built With

• api
• audio
• css
• mediapipe
• next.js
• react
• speech
• tailwind
• typescript
• web
• webassembly
• webmcp