Sehatsetu

Inspiration

Inspiration A hospital report can describe what is happening inside your body while leaving you unsure what the words mean—or even where the affected organ is. We created SehatSetu to bridge that gap. Building from India, we wanted language and unfamiliar medical terminology to become smaller barriers to understanding. Our goal is to help people learn about their reports and prepare better questions for their doctors. What it does SehatSetu connects medical-report findings to an interactive 3D anatomy atlas. Users can upload a text-readable PDF, explore associated organs, and ask follow-up questions. The viewer includes male and female anatomy models, selectable structures, organ isolation, and persistent highlights. Its interface supports English and seven Indian languages, with browser-based voice input and read-aloud features where supported. For the Alexa+ track, we are exploring a simulated conversational experience that connects report questions with visual anatomy. How we built it We extended an existing open-source anatomy viewer using React, TypeScript, and Three.js. A Python FastAPI backend extracts PDF text with PyMuPDF and processes findings through deterministic rules or optional Groq-powered analysis. The frontend checks anatomical references against the loaded atlas before highlighting structures. Voice interaction uses the browser’s Web Speech API. Challenges we faced Medical findings do not always use the same names as anatomical structures. Mapping conditions to appropriate organs required handling synonyms, laterality, negation, and structures unavailable in the model. We also worked on keeping highlights consistent while users rotate, isolate, or switch views. Multilingual interaction introduced additional challenges because speech recognition and available voices vary across browsers. What we learned A useful explanation needs context: the report wording, the relevant anatomy, and clear limits on what the visualization represents. Reference anatomy can help explain a finding, but it cannot reproduce a patient’s scan or establish a diagnosis. SehatSetu is an educational companion. Our next steps are improving multilingual responses, validating medical mappings, and implementing a complete Alexa+ integration.