Inspiration

Osteoporosis is frequently diagnosed only after a patient suffers a fragility fracture. As a radiologist, I routinely review conventional spine radiographs obtained for many different clinical reasons. Those studies, together with a small set of clinical risk factors, may provide an opportunity to identify patients who could benefit from further osteoporosis assessment before a fracture occurs.

OsteoScreen-AI was created to explore how existing radiographs can become a practical screening opportunity while keeping physicians fully in control of the final decision.

What it does

OsteoScreen-AI is a physician-in-the-loop clinical decision support prototype.

It combines clinical risk factors with conventional spine radiographs to generate a deterministic osteoporosis risk assessment. The workflow includes image analysis, Grad-CAM visual explanations, structured GPT-5.6 clinical reasoning, and final physician validation.

The key design principle is simple:

  • The deterministic engine calculates the result.
  • GPT-5.6 explains the result.
  • The physician makes the final clinical decision.

How we built it

The application was developed in Django using a modular architecture.

Clinical information and radiographic findings are processed by a deterministic engine. Image analysis combines conventional computer vision techniques with an optional MONAI model and Grad-CAM visualization to improve explainability.

After the deterministic result has been generated, GPT-5.6 receives only structured, de-identified information through the OpenAI Responses API using Structured Outputs. GPT never calculates the score or changes the recommendation; it only generates a structured clinical explanation.

Throughout development, Codex acted as an engineering collaborator. It accelerated implementation, helped review the architecture, refined prompts, audited the repository, validated integrations, and improved documentation while keeping the project's architectural decisions under human control.

Challenges we ran into

The biggest challenge was designing a system where generative AI could assist clinicians without becoming responsible for clinical decision-making.

We deliberately separated deterministic risk calculation from GPT-generated explanations to preserve reproducibility, transparency, and physician oversight. Integrating multiple AI components while maintaining clear architectural boundaries required continuous refinement.

Accomplishments that we're proud of

  • End-to-end physician-in-the-loop workflow.
  • Deterministic decision engine separated from generative AI.
  • Explainable image analysis using Grad-CAM.
  • Safe integration of GPT-5.6 through Structured Outputs.
  • Complete working prototype developed during OpenAI Build Week.

What we learned

Building trustworthy clinical AI depends as much on software architecture and safety as on model performance.

We learned the importance of separating deterministic reasoning from language generation, maintaining transparent workflows, and ensuring that physicians remain the final decision makers. We also experienced how Codex can significantly accelerate software development while preserving human oversight.

What's next for OsteoScreen-AI

Future work includes validating the approach on larger clinical datasets, improving the image models, integrating with hospital PACS/RIS environments, conducting prospective clinical evaluations, and refining physician-facing workflows before considering any real-world clinical deployment.

Built With

Share this project:

Updates