Safe-Scroll

Enable battlefield surgeons to get actionable patient info quickly to maximize medical outcomes.

Comment

Inspiration

The inspiration for Safe-Scroll came directly from the "D-Day (Survival)" theme. We thought about a battlefield operating theater. In these high-stakes environments, surgeons are under immense pressure to save lives but accessing patient info without breaking sterility is a big bottleneck.

While medical technology has advanced, the interface between the surgeon and the data hasn't. Touching a screen or a keyboard is a major contamination risk. We wanted to enable surgeons to access vital patient history and medical imaging using the "Audio" theme (voice/listening) and gesture controls, ensuring they can focus on the survival of their patients without compromising hygiene.

What it does

Safe-Scroll is a sterile, hands-free medical dashboard designed for battlefield surgeons.

  • Hands-Free Navigation: Using computer vision, the surgeon can navigate the entire UI using simple gestures with just one hand. This allows them to scroll through patient lists and select files without ever touching a device.
  • AI Patient Summaries: Upon selecting a patient, the system utilizes Google Gemini to generate a concise, medically relevant summary of the patient's condition. This summary is then read aloud using ElevenLabs text-to-speech, allowing the surgeon to get briefed audibly while keeping their eyes on the patient.
  • Imaging Viewer: Surgeons can view and navigate through 2D slices of 3D MRIs and CT scans, scrolling through layers of imaging data via gesture control.
  • Dynamic Data Management: All patient information, including logs and medical history, is securely stored and retrieved from a MongoDB database.

How we built it

  • Backend: We used MongoDB to structure our patient database to handle complex medical records and logs.
  • AI Integration: We used Google Gemini to process raw medical notes into digestible summaries. For the audio component, we integrated the ElevenLabs API to convert these summaries into clear, natural-sounding speech.
  • Interface: The frontend features a gesture-recognition engine that tracks the user's hand movements, translating them into scroll and click events. This was the key to making the app "hands-off."

Challenges we ran into

  • Gesture Sensitivity: Calibrating the "one-hand" navigation was difficult. We had to tune for responsiveness and avoid accidental inputs.

Accomplishments that we're proud of

  • True Sterility: We created a solution where a user can go from selecting a patient to viewing complex MRI slices and hearing a medical history without physically touching the computer.
  • Audio Integration: Using Gemini's intelligence with ElevenLabs' voice quality makes the AI voice genuinely helpful.
  • The UI: We designed a clean interface that is easy to read at a glance, which is crucial in high-stress environments.

What's next for Safe-Scroll

  • Real-Time Vitals: Integrating live feeds from heart rate monitors and ventilators directly into the dashboard.
  • Voice Commands: Adding two-way audio so the surgeon can dictate notes back into the MongoDB database during the procedure.
  • Prototype Testing: We will conduct thorough prototype testing in realistic medical scenarios to refine user experience and ensure Safe-Scroll meets the needs of battlefield surgeons effectively.
  • Partnerships: We are seeking collaborations with medical and military organizations to enhance Safe-Scroll's capabilities and further its development for real-world applications in crisis situations.

Built With

Share this project:

Updates