WhereItHurts

Patients show where it hurts in their body using the 3D Body map, and answer tailored AI questions. The GP gets it in medical terms, and every line links back to what the patient actually typed.​

Comment

Inspiration

A lot of GP care happens by phone now, and the doctor can't see where it hurts. The patient ends up describing a spot on their back from memory while the clock runs. One study found that clinicians interrupt patients about 11 seconds into their opening statement. The patient often gets off the call feeling unheard, and the doctor spends the first few minutes collecting basic facts.

We wanted to do that groundwork before the call. The patient says what's wrong in their own words, and the doctor starts the consult already knowing it. We also refused to let an AI make safety calls or put words in a patient's mouth.

What it does

Where It Hurts is a web check-in that the patient opens after booking a phone consult.

  • Patients mark their pain on a 3D body. They tap where it hurts, show where it spreads, and rate how bad it is. Nothing to describe in words.
  • A short adaptive chat. The AI asks 5–8 follow-up questions, but it can only choose them from a bank of 44 questions approved in advance by clinicians. It never writes its own questions, and it never diagnoses, advises or reassures.
  • Deterministic red-flag screening. Seven fixed rules (for example, new numbness around the groin, or new bladder or bowel trouble) check every answer before any AI runs. If a rule fires, the chat stops, the patient is told to call 000, and reception gets a red alert.
  • Patient recap. The patient sees a plain-language summary, can correct it, and confirms it.
  • A 20-second doctor panel. A side panel opens next to a mock practice-software screen. It shows red-flag status, a three-line headline, the body map snapshot, and "Clarify on the call" prompts. Every summary line links to the patient's exact words, and any line the system can't match to a quote is labelled Unverified instead of being hidden.
  • "Not asked" is not "No". The data and the UI keep "we didn't ask" separate from "the patient said no", so a gap never reads as a negative finding.
  • Reception queue. Staff see incoming check-ins live, with red-flag cases on top.

How we built it

  • Next.js 16 (App Router), React 19 and TypeScript in strict mode. Front end and back end are one repo with shared types.
  • Contracts first. In the first 90 minutes we wrote the shared types and the Zod schemas together. Every API input and output, and every JSON response from the LLM, is validated against them.
  • Two engineers, two lanes. Engineer A (the "Face") built the 3D body map, patient chat, recap, doctor panel and reception queue. Engineer B (the "Brain") built the conversation engine, question bank, red-flag rules, LLM provider, summary validator and API routes. Mock data that satisfies the same schemas let the UI get built before the engine existed.
  • 3D body map with three.js, @react-three/fiber and @react-three/drei. A tap returns the exact hit point, which is mapped to a body region.
  • Gemini sits behind a swappable LlmProvider interface and runs only in server routes, so API keys never reach the browser. The app has three modes: live, replay (cached responses, no network) and mock.
  • A quote validator checks every summary line against the transcript before the doctor sees it.
  • UI: Tailwind v4, shadcn/ui and Base UI, with Inter Tight and Petrona type. A WebGL fluid header adapted from Rare UI falls back to CSS if WebGL fails.
  • Workflow: Vitest unit tests, Playwright end-to-end tests, and GitHub Actions CI. Three Claude agents helped: a face builder, a brain builder, and a safety reviewer that checks every PR against our non-negotiables.
  • The judges' demo is a production build served through a Cloudflare Quick Tunnel.

Challenges we ran into

  • Keeping the AI in its lane. The model is useful for pulling facts out of messy patient speech, but we didn't trust it with safety. Red-flag rules are plain deterministic code that runs before every LLM call, and the prompts are limited to picking questions and extracting facts.
  • Traceability. The model sometimes paraphrases, which is how a summary line ends up saying something the patient never said. We built a validator that ties each line to an exact quote and visibly marks anything it can't verify.
  • Clinical language. Writing for patients and clinicians at once meant constant copy edits. Patient text is plain, second person and large; clinician text is concise and clinical. Neither is allowed to contain diagnosis or reassurance.
  • Messy LLM output. Invalid JSON and wrong shapes happen. Zod validation plus retries and fallback questions keep the chat running when the model misbehaves.
  • Bugs that only showed up on other machines. The animated header looked frozen on a teammate's laptop because of Windows reduced-motion settings, silent WebGL failures, and a fade that hid the colour. We made the motion bigger and added a CSS fallback. We also fixed hydration mismatches caused by dates and locales.
  • Two people moving fast in one repo. Folder ownership and frozen contracts kept merge conflicts rare. The ones we did hit taught us to rebase often.

Accomplishments that we're proud of

  • The full loop works end to end: tap the body, answer a few questions, confirm the recap, and the doctor's panel is ready.
  • Safety doesn't depend on the AI. The red-flag rules are testable code, and they run first every time.
  • Every line the doctor reads can be traced back to what the patient actually said.
  • We kept "Not asked" and "Patient said no" separate everywhere, which is easy to get wrong.
  • A clean split let two engineers and three AI agents work in parallel without breaking each other's code.
  • Everything uses synthetic data, and no secrets reach the browser.

What we learned

  • In healthcare software, the hard part is deciding what the AI is not allowed to do.
  • Freezing contracts early paid off quickly. Mocks meant nobody waited on anybody.
  • Small UI details carry a lot of trust. An "Unverified" label, a "Not asked" list, and a clear stop screen make the tool feel safe to use.
  • Test on other machines early. Something that looks fine on your laptop can look broken on a teammate's.
  • Working with AI coding agents goes better with strict ownership rules and a dedicated reviewer agent.

What's next for WhereItHurts

  • Finish the evaluation harness. Run 20 synthetic patient vignettes and publish red-flag recall and the rate of unsupported summary lines.
  • Clinical review of the question bank and red-flag rules, followed by a small supervised pilot with a GP practice.
  • Real integrations with practice software such as Best Practice and MedicalDirector, and with booking tools such as HotDoc, so the link goes out automatically and the summary lands in the patient's notes.
  • Cover more of the body. We started with musculoskeletal pain; next are other regions and presenting complaints.
  • More languages and voice input, so patients can check in however they're most comfortable.
  • Persistent storage and a hosted deployment in place of the local tunnel, plus accounts for multiple clinics.
  • Explore the regulatory pathway (TGA) before any real clinical use.

Built With

Share this project:

Updates

Submission history