RadiSense

dashboard
add x-ray
test outpout
test model
Arize Phoenix

Inspiration

What it does

How we built it

Inspiration

Millions of bone fractures are missed or delayed each year due to limited access to radiologists, especially in underserved areas. We wanted to build an AI agent that gives doctors faster, explainable, and trustworthy fracture analysis.

What it does

RadiSense is a multi-step AI clinical agent that:

Analyzes bone X-ray images using ResNet50 (98.79% accuracy)
Generates Grad-CAM visualizations showing WHERE the AI focused
Produces structured clinical reports using Gemini Flash
Traces every prediction with Arize Phoenix for full auditability

How we built it

ResNet50 trained on 9,246 X-ray images (Transfer Learning)
Gemini 2.5 Flash Lite for structured medical report generation
Arize Phoenix for real-time observability and tracing
Django REST API backend + Next.js frontend
Deployed on Hugging Face Spaces (backend) + Vercel (frontend)

Challenges

Memory optimization for PyTorch on free-tier hosting
Integrating Arize Phoenix tracing with Gemini API calls
Building a bilingual (FR/EN) clinical interface

What we learned

This project was a valuable learning experience.

One of the biggest lessons was understanding the importance of AI observability in medical applications. By integrating Arize Phoenix, we learned how to trace model predictions, monitor Gemini interactions, analyze latency, and improve transparency in AI workflows. This experience showed us that building trustworthy AI is not only about accuracy, but also about explainability and auditability.

We also gained practical experience in transfer learning with ResNet50, medical image analysis, Grad-CAM explainability, and AI-powered report generation using Gemini.

Another key takeaway was discovering the capabilities of Google Cloud and the Gemini ecosystem. Google AI Studio and Gemini models significantly accelerated development by enabling rapid experimentation, prompt engineering, and agent design. These tools made it easier to prototype intelligent workflows, build agentic systems, and integrate advanced reasoning capabilities into real-world applications.

Through this project, we learned that modern AI development is no longer limited to training models. By combining machine learning, foundation models, observability, and cloud technologies, it is possible to create powerful AI agents capable of solving complex real-world problems.

Overall, this project strengthened our skills in AI agents, machine learning, medical AI, explainable AI, observability, and full-stack AI application development.

Challenges we ran into

Building a medical AI agent on limited resources was one of our biggest challenges. We had to optimize the ResNet50 model to run efficiently on free-tier hosting while maintaining strong performance. As an AI Master's student, I wanted to explore more advanced architectures such as EfficientNet, but we chose ResNet50 because it could be trained and deployed effectively using the Google Colab T4 GPU resources available to us. Google's tools and infrastructure have been extremely valuable for our education and projects. If we receive support through this hackathon, we plan to upgrade our pipeline with more advanced models and continue improving and completing the project. Integrating Arize Phoenix tracing with Gemini workflows also required careful configuration to ensure complete observability across the entire pipeline. Another challenge was designing an intuitive interface that presents complex AI outputs in a clear and useful way for healthcare professionals.

Accomplishments that we're proud of

This project taught us that trustworthy AI requires more than accurate predictions. Through Arize Phoenix, we learned the importance of observability, monitoring, and auditability in medical AI systems. We also gained valuable experience with transfer learning, explainable AI using Grad-CAM, AI-powered report generation with Gemini, and full-stack AI application development. Google Cloud and the Gemini ecosystem showed us how modern AI tools can accelerate innovation and simplify the creation of intelligent agents capable of solving real-world challenges.

What's next for RadiSense

Our next goal is to expand RadiSense beyond fracture detection to support additional musculoskeletal conditions and other medical imaging modalities. We plan to improve the agent's reasoning capabilities, integrate more advanced clinical workflows, and enhance multilingual support. We also aim to deploy RadiSense in real-world healthcare environments, where explainable and observable AI can help clinicians make faster and more informed decisions.