Twain

We make clinical reasoning visible, so gaps that traditional exams miss can be identified and fixed. With a voice-first approach, we map students’ reasoning into mind maps to find gaps and study.

Comment

Inspiration

We are an unusual hackathon team: two medical interns and an attending physician. We've all deviated from the traditional medical path and have explored the intersections of healthcare with other disciplines: data science, education, informatics, digital health, innovation and artificial intelligence. Those overlapping interests brought us together years ago and ultimately led us to participate in this hackathon project.

At first, we wanted to explore something outside of the domains in which we have lived experience (in particular, medical education technology, where we have spent a few years building a suite of tools for universities). However, as we tried exploring new ideas, we realized that we would probably end up guessing user needs, and with little time, it made sense to take one of the unmet needs we had seen many times in the past but hadn't been able to work on because 1) the technology wasn't there yet and 2) there hadn't been a compelling reason to sit down and actually get it done.

That unmet need we came up with was a better way of practicing differential diagnosis skills in late-year medical students in order to improve their clinical reasoning - that is, the art of approaching a patient's chief complaint in order to come up with a diagnosis. Medical students spend years learning diseases, but real cases do not arrive as disease names or MCQs: patients arrive with symptoms.

It's the clinician's job to ask the right questions (not too many for time efficiency, but also not too few to miss a vital clue) to assess those symptoms, and to integrate the patient's history and findings with its internal model of the diseases and their manifestations in order to quickly narrow it down to set of few potential diagnosis. The first step in acquiring this skill set is creating an internal mental model, which we have found helpful to represent as mind maps. However, there's no easy way to practice these mind maps, and in reality, it's usually seasoned clinicians that teach these approach to students opportunistically over the years.

We have experienced this gap in medical training ourselves. Before rushing into building a set of features, we validated the need with 14 interviews of medical students, interns, residents and a few doctors and then ran a convenience survey of 85 chilean medical students (our target audience for Twain). In that sample, 68% reported recurrent difficulty building clinical reasoning from a chief complaint despite knowing related conditions, and 64% reported recurrent difficulty organizing diagnoses by categories. This was the signal that confirmed our interest in exploring if AI could help learners visualize their own reasoning, without doing the reasoning for them. More details about our approach to user research in https://withtwain.com/about.

What it does

Twain is a voice-first clinical reasoning application for medical students preparing for clinical rotations, oral exams, OSCE reasoning and case presentations overall.

In the current MVP, the learner receives one bounded task: to build a structured differential diagnosis for a presenting problem such as shortness of breath, chest pain or fever. They reason and answer out loud, and watch their mind map take shape on screen, as Twain maps their words to concepts in the nodes of the mind map.

Twain protects the students unaided first attempt, but can offer hints if asked, similar to a clinical teacher helping a student notice a missing connection without simply providing the complete answer. The system records which concepts were produced independently and which appeared only after assistance.

At the end of the drill, the learners get to compare the mind map they built, differentiating the components they recovered themselves from the ones they recovered with a clue, versus the ones they forgot to mention. Across repeated attempts, Twain builds an individual heat map showing how consistently different concepts are retrieved by the student.

We also built a cohort based system for instructors. It aggregates activity, completion, concept coverage, and group heat maps, to measure across an entire class how their clinical reasoning skills are evolving across a set of clinical scenarios. We envision this would build valuable datasets for improved lecturing, one-on-one tutoring but also clinical program accreditation, as this is real evidence of the attainment of harder-to-assess learning outcomes, such as clinical reasoning.

How we built it

Codex and GPT-5.6 were a fundamental member of our team for this project. We began by brainstorming with the team, and then had GPT-5.6 Pro help us redact interview guides to use during the first round of user research. As we started developing the concept, we also used ChatGPT to criticize and help us refine our understanding, and run deep research and competitor validation to make sure our integrated approach was novel. Besides all the building, Codex and GPT 5.6 Sol was very helpful in processing and labeling our fourteen user interviews and 85 survey responses, as well as analyzing the application analytics from Posthog.

Besides actively participating in the idea and concept behind Twain, Codex and GPT 5.6 Sol helped us build Twain as a full-stack TypeScript application designed around the previously described core experience as opposed to a chatbot. We intentionally decided to privilege the voice-in / visual-out data stream as we believe (and wanted to provide evidence) of the benefits of these modalities in terms of information bandwidth and improved user experience over different input modalities (e.g., writing or selecting / dragging from a list).

The interface is built with React, Vite and React Flow. A fustily backend owns the learning state, validates transitions, and determines what appears on the map. Supabase provides auth and a hosted PostgreSQL database, with Drizzle managing the data model.

For voice practice, we used Deepgram Nova-3, as it produces streaming transcripts and we found it to be the most cost-efficient for our use case (they provided free credits) and it also performed reasonably well in Spanish and with medical terms. Rule-based recognition handle reviewed labels, acceptable synonyms (e.g., abbreviations of conditions, such as A Fib instead of Atrial Fibrillation) and common transcription variants. When language is ambiguous, we use GPT 5.6 Luna via API with Structured Outputs to propose a mapping from the learner's words to a bounded set of allowed clinical concepts. The server then validates the exact quote provided, concept, relationship and overall state before accepting it.

We built English and Spanish support, as we wanted to have in person user testing with real students. We also built in text fallback (currently not deployed), review snapshots, individual progress views and role-restricted cohort dashboards and a Mind Map Management System (MMMS) to edit existing mind maps or create new ones from scratch. To facilitate that progress, we integrated GPT 5.6 Sol via the API with structured outputs to have it seed a first iteration of a Mind Map given a title and an optional reference structure. We tried a similar approach having Codex seed the data model directly, which also worked great.

Finally, we added a product analytics layer on top using Posthog, and made sure to consider the privacy considerations in our terms of use and analytics documents, available on the website.

Challenges we ran into

Some of the most complex problems we found had to do with assertion management, including cases such as differentiating a question versus a positive statement; or negations. Streaming voice, which we felt was fundamental to avoid repeating themselves, added a new set of challenges, related to the processing of partial transcripts that change while the user speaks, or the interpretation of pauses.

Another problem we found with our initial naive approach to hints is that the LLM would give out the answer in trying to come up with a hint. We decided to author specific hints for each disease and have the system recognize when the user asked for a hint, only providing a controlled hint that avoids leaking the answers.

Another challenge worth mentioning had to do with the actual content of the application, the mind maps themselves. We knew we needed good quality materials so we could go beyond a simple demo, which is a problem we've experienced before building medical educational technology: once you've solved the platform issue, getting good content becomes the bottleneck.

To solve this, we reached out to Andre Mansoor, MD from OHSU, a well known Internal Medicine physician known for his Frameworks for Internal Medicine, and asked him if we could adapt some of the mind maps in that work. Dr. Mansoor was kind enough to give us consent to do so, which allowed us to focus on finalizing the technical features before trying out the MVP on our first users.

Accomplishments that we're proud of

We are proud we were able to create and end-to-end functioning learning system rather than a voice demo layered on top a static mind map.

Twain currently allows for new mind maps to be created from the UI, and beside the core student features (the main MVP), it has a functional institutional layer to allow instructors to inspect cohort-level activity and concept-retrieval maps.

We're also proud we were able to root our approach in real user research, and that we were able to get the system ready for testing in time for actual product metrics. At the time of submission, we had and has a functioning user system that has actual students from chilean Universities trying it out in a closed beta (more of that later).

What we learned

Building Twain taught us that voice-first AI is much more than recording and transcribing audio: we learned to process audio in real time and how to use WebRTC for low-latency bidirectional communication. One surprisingly difficult challenge was utterance segmentation while managing partial transcripts, corrections, retries, and duplicate events without turning provisional speech into evidence.

Most importantly, we learned that AI in education often requires restraint. An unconstrained model can generate an excellent differential diagnosis as a hint, but then it has performed the reasoning for the student. It can also create very intricate mind maps, but then they have really limited utility for the students.

Twain therefore uses AI to recognize and organize what the learner expressed, while reviewed clinical content and deterministic rules control hints, progression, and feedback.

We also learned a bit about AI video generation and different platforms. We wanted to tell a story in a way that any listener could empathize, and feel that even with its quirks, AI generated video is allowing people like us to tell our stories better.

Finally, we learned that the Mark Twain quote that inspired us to build and name this application ("It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so.") isn't really attributable to him, which is perfect to the quote and the Twain app itself. In medicine, it's usually thinking you are ready because know all the diseases by heart what gets you in trouble. We left a couple of easter eggs on the app related to this.

What's next for Twain

Twain is currently live in a closed beta with students from Chilean universities. During the first 24 hours, we've already seen a first signal. In the first 24 hours of our closed beta, 46 unique visitors reached the app, 24 signed in (52%), 12 opened the drill library, and 8 started a voice practice session (33% of those that signed in). What matters most to us is what happened after the start: 75% of those who started, finished the drill, and 50% of those who finished one drill did a second one. There are clear improvements to be made on the onboarding to increase the amount of people who log in that actually attempt a drill, but we believe these first numbers are early indicators that unaided-first practice, which students in our survey admitted they avoid, becomes something they might voluntarily repeat when the feedback is evidence of where to improve instead of a grade. One pilot student said about Twain: "It shows me which concepts I still haven't integrated: it puts my head in order", and we got 1 student that signed up to be notified when paid subscriptions will be available in the future.

We are freezing development today and are excited to see what comes out of the closed beta, as well as to see the results of the Hackathon.

Built With

Share this project:

Updates