Inspiration Artificial intelligence has significant potential to support medical image analysis, but understanding how an AI model reaches its predictions remains a major challenge. We were inspired to build XRayLens AI to explore how AI can become more transparent, interpretable, and easier to evaluate. Instead of treating a model’s prediction as a black box, we wanted users to investigate why a chest X-ray might be classified as normal or showing signs of pneumonia.
What it does XRayLens AI is an interactive, browser-based research and educational prototype for exploring AI-powered chest X-ray classification. Its key features include:
AI classification: Produces predictions for normal and pneumonia categories when the model is available. Grad-CAM visualization: Generates heatmaps that help users explore which image regions influenced a model’s prediction. Interactive heatmap controls: Allows users to adjust visualization opacity and select the class being explained. Robustness testing: Applies image modifications to investigate how consistently the model responds to changes in the input. Reliability and provenance: Provides additional context for evaluating model behavior, experimental evidence, and limitations. The goal is to help users understand not only what an AI model predicts, but also how its predictions can be examined and questioned.
How we built it We built XRayLens AI as a web-based interactive prototype, with an interface designed to make machine learning concepts accessible. The application brings together chest X-ray selection, classification outputs, visual explanations, and robustness testing in one workflow.
The project incorporates the concept of Grad-CAM to visualize regions influencing a selected class score. It also includes stress-testing functionality intended to compare predictions before and after image transformations. The interface organizes these capabilities into distinct sections so users can explore the model step by step.
Challenges we ran into One of our main challenges was making a complex machine learning workflow understandable through a simple interface. Visual explanations can be helpful, but they must be presented carefully so users do not mistake a heatmap for proof that the model is reasoning correctly.
Another challenge was considering model reliability. A prediction may change because of image characteristics that are not medically meaningful, so we wanted to provide a way to investigate prediction stability rather than relying on a single output.
We also had to consider the limitations of AI in medical imaging, including the possibility of incorrect predictions, model availability, and the importance of responsible data handling.
Accomplishments that we're proud of We are proud to have brought several important AI concepts together in one interactive prototype. XRayLens AI combines image classification, visual explainability, and robustness testing in a single user experience.
We are especially proud of the project's focus on transparency. Rather than presenting AI as an unquestionable authority, the prototype encourages users to explore its outputs, examine its behavior, and think critically about its limitations.
What we learned Through this project, we learned that building a useful AI application involves more than generating predictions. Interpretability, reliability, user experience, and responsible communication are also essential.
We explored how Grad-CAM can help visualize model behavior and how stress testing can reveal potential sensitivity to image changes. We also learned that visual explanations have limitations and that a model's output should not automatically be treated as correct or clinically meaningful.
Most importantly, we learned the value of designing AI tools that encourage critical thinking rather than blind trust.
What's next for XRayLens AI Our next steps include improving model integration and reliability, expanding robustness tests, and making the explanation tools more informative. We would also like to explore additional evaluation metrics, improve documentation of the model and its training data, and test the prototype against a broader range of appropriately sourced, de-identified research images.
In the future, we hope to make XRayLens AI a more comprehensive educational platform for understanding explainable AI in medical imaging, while maintaining a strong focus on transparency, privacy, and responsible use.
XRayLens AI is a research and educational prototype, not a medical diagnostic device, and should not be used to make healthcare decisions.
Log in or sign up for Devpost to join the conversation.