XrayVision

Inspiration

Hack the world a better place! - This motto drives our vision to make medical imaging more accessible, trustworthy, and reassuring for everyone, regardless of their background. Around the world, many patients - especially in less wealthy areas - face long waits for expert analysis, leading to anxiety and uncertainty. By using multiple AI models, we provide an instant, diverse opinions, increasing trust and transparency in AI-assisted healthcare. We aim to support the UN goals for health (Goal 3) and equality (Goal 10) by ensuring that people from all demographics, have better access to medical insights. We aim to empower patients with clarity, multiple perspectives, and a greater sense of safety in their healthcare journey.

What it does

XrayVision is a web app that empowers patients by providing instant diagnostic insights from multiple AI models. By leveraging an ecosystem of diverse AI models, we ensure greater trust, reduced uncertainty, and more reliable results compared to a single-model approach. Our platform not only detects diseases and pathologies but also offers AI-driven explanations, generates comprehensive reports, and provides personalized recommendations for next steps. It’s designed to be a supportive tool for gaining feedback or a second opinion.

How we built it

Our team developed XrayVision by combining cutting-edge AI technologies with robust software engineering principles to create a seamless and reliable platform. The front-end and back-end are built in Python using Streamlit, ensuring a user-friendly and efficient interface. We integrated multiple state-of-the-art foundation models (FM), trained on billions of text and image pairs—including models from Mistral-AI and Perplexity—to enhance diagnostic accuracy and trust. To power our AI models, we leverage the NVIDIA GeForce RTX 3090 (24GB), enabling high-performance inference for real-time disease detection and explanation.

Challenges we ran into

Balancing high performance and real-time processing capabilities without compromising user experience was complex. We are new to front-end development, so creating a front-end which shows all features in a nice way took more time than expected.
Combining models that have different environment requirements

Accomplishments that we're proud of

• our teamwork and especially the result of our project.
• the development of the first patient-focused app that integrates an ecosystem of multiple foundation models to detect diseases from chest x-rays, but even more important help a patient to understand complicated reports and diagnosis
• We could test our platform on a friend's x-ray and IT WORKED -> disease detection and localization :)
• Contributing to global health with cutting edge technologies as our platform is easy accessible on any device.

What we learned

• Teamwork makes the dream work! <3
• Push & Pull even more often if you work at the same time on the same project.

What's next for XrayVision

While XrayVision is currently focusing on analysis of chest x-rays, the next step will go one step further from 2D to 3D, we want to include CT and MRI disease analysis.

Built With

• api
• app
• chexagent
• chexpert
• edge-ai
• foundational-models
• llms
• medimageinsight
• mimic
• mistral
• perplexity
• python
• vlm