SpeedRay: Accelerating Chest X-Ray Diagnosis

• 

  This is the layout of what the X-ray analysis and reporting look like.

Inspiration

Living in Canada, we are intimately familiar with the crippling wait times in our emergency rooms. A massive, hidden bottleneck in this system is diagnostic imaging. Radiologists are facing unprecedented burnout, often required to read hundreds of X-rays in a single, exhausting shift. When a physician is experiencing cognitive fatigue, reading a scan takes longer, and the backlog grows.

We realized that the solution isn't to replace doctors with AI—it’s to rescue them from cognitive overload and administrative burden. We were inspired to build a "second set of eyes" for our strained healthcare system. We wanted to create a tool that handles the repetitive, time-consuming parts of radiology, allowing the human experts to focus entirely on final decision-making and patient care.

What it does

Our platform is an AI-powered diagnostic designed to eliminate hospital workflow bottlenecks. It acts as an intelligent bridge between the moment a chest X-ray is taken and the moment the doctor reviews it. Here is how the seamless workflow operates: Instant Pre-Analysis & Triages: As soon as the patient's chest X-ray photo is uploaded, our computer vision analyzes and annotates the scan. It detects anomalies—such as signs of pneumonia—and visually highlights the specific regions of concern on the image. Automated Medical Reporting: Instead of forcing the doctor to start from a blank page, the AI translates its visual findings into a drafted, professional medical report.

Human-in-the-Loop Confirmation: The radiologist opens the case to find the heavy lifting already done. They review the highlighted X-ray, read the AI's baseline diagnosis, and use their medical expertise to either confirm, edit, or reject the findings. By teeing up the evidence and drafting the initial paperwork, we turn a manual, multi-minute review process into a 30-second expert confirmation. We drastically speed up diagnosis times and reduce physician burnout, all while keeping the human expert firmly in the driver's seat.

How we built it

Initially, I hypertuned the code for the prebuilt torchxrayvision(pytorch X-ray) model to identify 14 clinical diseases from Kaggle dataset (NIH Chest X-ray model) from https://www.kaggle.com/datasets/nih-chest-xrays/data. Later on, I utilized Cloudinary to annotate the disease and risk classification with name and highlight them with a specific colour indicating severity level calculated from critical value from the hypertuned PyTorch model. (Low: green, score <= 0.5, Medium: yellow, 0.5 < score <= 0.62, High: red, score > 0.62). The one in which out of the 14 clinical disease has the highest anomaly score would be the diagnosed disease based on global confidence threshold. I used LangChain RAG from Gemini or OpenAI API to map X-ray finding with the associated interpretation of the X-ray disease in medical context. Gemini or OpenAI would support determinstic langchain for a more exact written result of the X-ray report. Elevenlab was used not only to hear the medical interpretation of the disease associated from particular X-ray (TTS) but to bring a mini conversation between the doctor about understanding how the image annotation and result from the report are connected through the image finding stated on the report and he can agree or diagree. If the doctor thinks the X-ray interpretation and report are correct, he would submit else it would allow him to edit the written report manually or have AI to reinterpret and Cloudinary to re-annotate the image through Refresh annotation. Google Antigravity was used to set a template for Langchain to generate a structured report and to develop the initial template of the frontend of the code with React and typescript and orchestration of the architecture. Cursorcode is used to refine the orchestration quality and support the backend structure of fastAPI. Backboard.io was used to develop AI pipeline and to provide persistent memory onto Langchain RAG. NoSQL was used store information from X-ray analysis and for RAG-medical knowledge storage. Auth0 is used for user-authentication to secure private data, and Vultr hosted the backend to reliably connects the frontend via internet, and idle-shutdown setup turns on when the frontend isn't used, so I am losing less credits.

Challenges we ran into

In the initial phase, one of biggest challenges I faced with tuning the hyperparameter model was false positive. Initially on a normal X-ray testing, all diseases had a global confidence threshold variable > 0.5 even though the image was actually normal condition. I had to find the sweet spot to determine the threshold variable such that there are less false positive nor false negative. The other challenges was the report cannot be generated due to gemini 1.5 being an outdated model. However, I referred to gemini-2.0-flash-pro model. However, its quota quickly depleted. I had to resort to MLH website to reset $300 free credit. Still, gemini-2.0-flash-pro quota was lost. As a solution, I developed a fallback condition. If gemini-2.0-flash-pro or gemini-1.5 failed then use all other models in Gemini libraries and OpenAI libraries. Finally, the challenge I had was setting up Vultr, specifically steps to claim gift code. I read through MLH instruction to claim $100 free credit. However, some of the API calls failed due to setting up hardware and transferring data to run the app. For example, ElevenAPI keys were lost. I had to resort to CursorCode in debug mode to resolve all dependency and backend API issues with the code. This allowed the backend to connected reliably with the frontend via internet, the logic were the same. I also had to deal with losing credits per hour. As a solution, I developed a middleware to help stop the server from running when it is idle (meaning frontend did not appear in the browser).

Accomplishments that we're proud of

I integrated a cloud environment to my project that runs simulatenously with the local environment. I was able to orchestrate a project architecture and build an AI pipeline.

What we learned

I learned how to utilize IaaS (Infrastructure as a code) platform to support my app. Additionally, I have learned how to persist storage memory across all pipeline and orchestration of the code and how to utilize RAG and hypertune and adjust a pre-trained model.

What's next for Untitled

When it comes to reload annotation, develop an AI pipeline from reinterpreting the X-ray image and re-annotating to rewriting the entire report if radiologist feels the information is incorrect. I could consider incorporating Tailscale to see if my app can be tested reliably not only on web but also on a mobile app and to encrypt in transit private information. I could train and incorporate more set of diseases into the training model and make the annotation more precise to specific anomaly detected location from an X-ray photo. I would consider other types of X-ray to train simultaneously (e.g. spine, abdomen, etc). Maybe require nurses to upload requisition for clear communication between different parties in a fast paced environment. Maybe connecting with the hospital's X-ray software APIs so that my software can easily retrieve their X-ray photo and pre-analyze them and generate initial summary. Deleting would be fictional data would be crucial and have it audited by 3rd party before use by real patient data.

Built With

• auth0
• backboard.io
• cloudinary
• cursorcode
• elevenlab
• fastapi
• geminiapi
• googleantigravity
• langchain
• nosql
• openaiapi
• rag
• react
• torchxrayvision
• typescript
• vultr