PulseLens: Multimodal Tele-Triage Assistant

Inspiration

In under-resourced community health centers and emergency outpatient desks, medical personnel face critical intake backlogs. Clinicians and triage staff spend crucial minutes gathering subjective narratives, inspecting wounds or visible trauma, and manually drafting clinical intake records. Delays at this entry point increase the risk of acute conditions escalating unnoticed. We built PulseLens to serve as an edge-ready tele-triage co-pilot that rapidly structures intake observations into standardized clinical workflows.

What it does

PulseLens processes multimodal patient evidence—combining reported symptoms, recorded vitals, and clinical imagery (such as dermatology lesions, burns, or lacerations)—to provide instantaneous clinical decision support:

• Triage Stratification: Automatically categorizes patient acuity into structured tiers (Immediate, Semi-Urgent, or Routine Primary Care).
• Automated SOAP Notes: Generates structured Subjective, Objective, Assessment, and Plan (SOAP) clinical documentation ready for clinician review.
• Critical Risk Highlighting: Flags urgent complications (such as signs of infection, hypoxia, or tissue trauma) before formal examination.

How we built it

• Interface & Workflow: Developed with Python and Streamlit to ensure a lightweight, low-latency UI that runs smoothly on low-bandwidth setups.
• Multimodal Visual Reasoning: Integrated Google Gemini to simultaneously parse raw textual history and high-resolution clinical photographs, returning structured, parseable clinical summaries.
• Structured Data Handling: Enforced consistent medical documentation schemas to standardize terminology across pre-screening steps.

Challenges we faced

• Ensuring balanced and safe clinical outputs: Tuning the model prompt structure so it acts strictly as a clinician-assistive tool rather than providing direct diagnostic claims.
• Structuring multimodal inputs cleanly within Streamlit to handle variable photo sizes and missing vital values gracefully without pipeline crashes.

Accomplishments that we're proud of

• Delivering a zero-friction, end-to-end tele-triage workflow that transforms an injury photo and symptom bullet points into a clean SOAP note in seconds.
• Keeping the architecture lightweight and responsive enough for field deployment in primary clinics with limited local compute infrastructure.

What we learned

• Prompt design techniques for reliable clinical schema extraction in high-stakes workflows.
• Managing fast-turnaround UI state updates alongside multimodal API calls in Streamlit.

What's next for PulseLens

• Direct export integrations into open-standard Electronic Health Records (EHR / FHIR formats).
• Offline-first on-device lightweight model fallback for areas experiencing complete network outages.

Built With

• computer-vision
• gemini-api
• google-gemini
• multimodal
• pillow
• python
• streamlit